[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk1.8.0_144) - Build # 1222 - Failure!

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk1.8.0_144) - Build # 1222 - Failure!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1222/
Java: 64bit/jdk1.8.0_144 -XX:-UseCompressedOops -XX:+UseG1GC

2 tests failed.
FAILED:  org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly

Error Message:
Unexpected number of elements in the group for intGSF: 4

Stack Trace:
java.lang.AssertionError: Unexpected number of elements in the group for intGSF: 4
        at __randomizedtesting.SeedInfo.seed([F2644420F7092010:69DF2A78BA51124E]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly(DocValuesNotIndexedTest.java:377)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory

Error Message:
expected:<5> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
        at __randomizedtesting.SeedInfo.seed([F2644420F7092010:9F98E0DD4D41DF17]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:265)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12772 lines...]
   [junit4] Suite: org.apache.solr.cloud.DocValuesNotIndexedTest
   [junit4]   2> 1135853 INFO  (SUITE-DocValuesNotIndexedTest-seed#[F2644420F7092010]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.DocValuesNotIndexedTest_F2644420F7092010-001/init-core-data-001
   [junit4]   2> 1135854 WARN  (SUITE-DocValuesNotIndexedTest-seed#[F2644420F7092010]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
   [junit4]   2> 1135854 INFO  (SUITE-DocValuesNotIndexedTest-seed#[F2644420F7092010]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1135856 INFO  (SUITE-DocValuesNotIndexedTest-seed#[F2644420F7092010]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 1135856 INFO  (SUITE-DocValuesNotIndexedTest-seed#[F2644420F7092010]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.DocValuesNotIndexedTest_F2644420F7092010-001/tempDir-001
   [junit4]   2> 1135856 INFO  (SUITE-DocValuesNotIndexedTest-seed#[F2644420F7092010]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1135856 INFO  (Thread-4299) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1135856 INFO  (Thread-4299) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1135868 ERROR (Thread-4299) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1135956 INFO  (SUITE-DocValuesNotIndexedTest-seed#[F2644420F7092010]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:45479
   [junit4]   2> 1135958 INFO  (zkConnectionManagerCallback-2223-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1135960 INFO  (jetty-launcher-2220-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1135960 INFO  (jetty-launcher-2220-thread-2) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1135965 INFO  (jetty-launcher-2220-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1135965 INFO  (jetty-launcher-2220-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1135965 INFO  (jetty-launcher-2220-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1135965 INFO  (jetty-launcher-2220-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1135965 INFO  (jetty-launcher-2220-thread-1) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1135965 INFO  (jetty-launcher-2220-thread-2) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1135966 INFO  (jetty-launcher-2220-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4f4898af{/solr,null,AVAILABLE}
   [junit4]   2> 1135966 INFO  (jetty-launcher-2220-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3cfbc7db{/solr,null,AVAILABLE}
   [junit4]   2> 1135967 INFO  (jetty-launcher-2220-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@6e723732{HTTP/1.1,[http/1.1]}{127.0.0.1:33383}
   [junit4]   2> 1135967 INFO  (jetty-launcher-2220-thread-2) [    ] o.e.j.s.Server Started @1137409ms
   [junit4]   2> 1135967 INFO  (jetty-launcher-2220-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@74deb9a9{HTTP/1.1,[http/1.1]}{127.0.0.1:33027}
   [junit4]   2> 1135967 INFO  (jetty-launcher-2220-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33383}
   [junit4]   2> 1135967 INFO  (jetty-launcher-2220-thread-1) [    ] o.e.j.s.Server Started @1137409ms
   [junit4]   2> 1135967 INFO  (jetty-launcher-2220-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33027}
   [junit4]   2> 1135967 ERROR (jetty-launcher-2220-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1135967 ERROR (jetty-launcher-2220-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1135967 INFO  (jetty-launcher-2220-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.3.0
   [junit4]   2> 1135967 INFO  (jetty-launcher-2220-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.3.0
   [junit4]   2> 1135967 INFO  (jetty-launcher-2220-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1135967 INFO  (jetty-launcher-2220-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1135967 INFO  (jetty-launcher-2220-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1135967 INFO  (jetty-launcher-2220-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1135967 INFO  (jetty-launcher-2220-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-22T16:32:05.362Z
   [junit4]   2> 1135967 INFO  (jetty-launcher-2220-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-22T16:32:05.362Z
   [junit4]   2> 1135968 INFO  (zkConnectionManagerCallback-2226-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1135968 INFO  (zkConnectionManagerCallback-2227-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1135968 INFO  (jetty-launcher-2220-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1135968 INFO  (jetty-launcher-2220-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1135975 INFO  (jetty-launcher-2220-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45479/solr
   [junit4]   2> 1135975 INFO  (jetty-launcher-2220-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45479/solr
   [junit4]   2> 1135976 INFO  (zkConnectionManagerCallback-2233-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1135976 INFO  (zkConnectionManagerCallback-2235-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1135977 INFO  (zkConnectionManagerCallback-2237-thread-1-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1135978 INFO  (zkConnectionManagerCallback-2239-thread-1-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1136007 INFO  (jetty-launcher-2220-thread-2) [n:127.0.0.1:33383_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1136007 INFO  (jetty-launcher-2220-thread-1) [n:127.0.0.1:33027_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1136008 INFO  (jetty-launcher-2220-thread-2) [n:127.0.0.1:33383_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33383_solr
   [junit4]   2> 1136008 INFO  (jetty-launcher-2220-thread-1) [n:127.0.0.1:33027_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33027_solr
   [junit4]   2> 1136008 INFO  (jetty-launcher-2220-thread-2) [n:127.0.0.1:33383_solr    ] o.a.s.c.Overseer Overseer (id=99394435498442757-127.0.0.1:33383_solr-n_0000000000) starting
   [junit4]   2> 1136010 INFO  (zkCallback-2236-thread-1-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1136010 INFO  (zkCallback-2238-thread-1-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1136015 INFO  (jetty-launcher-2220-thread-2) [n:127.0.0.1:33383_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33383_solr
   [junit4]   2> 1136015 INFO  (zkCallback-2236-thread-1-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1136015 INFO  (zkCallback-2238-thread-1-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1136053 INFO  (jetty-launcher-2220-thread-1) [n:127.0.0.1:33027_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33027.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446f7595
   [junit4]   2> 1136058 INFO  (jetty-launcher-2220-thread-1) [n:127.0.0.1:33027_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33027.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446f7595
   [junit4]   2> 1136058 INFO  (jetty-launcher-2220-thread-1) [n:127.0.0.1:33027_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33027.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446f7595
   [junit4]   2> 1136059 INFO  (jetty-launcher-2220-thread-1) [n:127.0.0.1:33027_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.DocValuesNotIndexedTest_F2644420F7092010-001/tempDir-001/node1/.
   [junit4]   2> 1136072 INFO  (jetty-launcher-2220-thread-2) [n:127.0.0.1:33383_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33383.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446f7595
   [junit4]   2> 1136077 INFO  (jetty-launcher-2220-thread-2) [n:127.0.0.1:33383_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33383.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446f7595
   [junit4]   2> 1136077 INFO  (jetty-launcher-2220-thread-2) [n:127.0.0.1:33383_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33383.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446f7595
   [junit4]   2> 1136078 INFO  (jetty-launcher-2220-thread-2) [n:127.0.0.1:33383_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.DocValuesNotIndexedTest_F2644420F7092010-001/tempDir-001/node2/.
   [junit4]   2> 1136108 INFO  (zkConnectionManagerCallback-2245-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1136110 INFO  (zkConnectionManagerCallback-2249-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1136111 INFO  (SUITE-DocValuesNotIndexedTest-seed#[F2644420F7092010]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1136111 INFO  (SUITE-DocValuesNotIndexedTest-seed#[F2644420F7092010]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45479/solr ready
   [junit4]   2> 1136113 INFO  (qtp1396299069-12187) [n:127.0.0.1:33027_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&maxShardsPerNode=2&name=dv_coll&nrtReplicas=1&action=CREATE&numShards=4&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1136114 INFO  (OverseerThreadFactory-4487-thread-1-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection dv_coll
   [junit4]   2> 1136217 INFO  (OverseerStateUpdate-99394435498442757-127.0.0.1:33383_solr-n_0000000000) [n:127.0.0.1:33383_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"dv_coll",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"dv_coll_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:33383/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1136217 INFO  (OverseerStateUpdate-99394435498442757-127.0.0.1:33383_solr-n_0000000000) [n:127.0.0.1:33383_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"dv_coll",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"dv_coll_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:33027/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1136218 INFO  (OverseerStateUpdate-99394435498442757-127.0.0.1:33383_solr-n_0000000000) [n:127.0.0.1:33383_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"dv_coll",
   [junit4]   2>   "shard":"shard3",
   [junit4]   2>   "core":"dv_coll_shard3_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:33383/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1136219 INFO  (OverseerStateUpdate-99394435498442757-127.0.0.1:33383_solr-n_0000000000) [n:127.0.0.1:33383_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"dv_coll",
   [junit4]   2>   "shard":"shard4",
   [junit4]   2>   "core":"dv_coll_shard4_replica_n6",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:33027/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1136421 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=dv_coll_shard2_replica_n2&action=CREATE&numShards=4&collection=dv_coll&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1136421 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1136434 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=dv_coll_shard1_replica_n1&action=CREATE&numShards=4&collection=dv_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1136434 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1136441 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=dv_coll_shard3_replica_n4&action=CREATE&numShards=4&collection=dv_coll&shard=shard3&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1136441 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=dv_coll_shard4_replica_n6&action=CREATE&numShards=4&collection=dv_coll&shard=shard4&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1136544 INFO  (zkCallback-2238-thread-1-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1136544 INFO  (zkCallback-2236-thread-1-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1136544 INFO  (zkCallback-2238-thread-2-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1136544 INFO  (zkCallback-2236-thread-2-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1137446 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1137452 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.IndexSchema [dv_coll_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 1137455 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1137455 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'dv_coll_shard2_replica_n2' using configuration from collection dv_coll, trusted=true
   [junit4]   2> 1137455 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33027.solr.core.dv_coll.shard2.replica_n2' (registry 'solr.core.dv_coll.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446f7595
   [junit4]   2> 1137456 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1137456 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SolrCore [[dv_coll_shard2_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.DocValuesNotIndexedTest_F2644420F7092010-001/tempDir-001/node1/dv_coll_shard2_replica_n2], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.DocValuesNotIndexedTest_F2644420F7092010-001/tempDir-001/node1/./dv_coll_shard2_replica_n2/data/]
   [junit4]   2> 1137457 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1137464 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1137480 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.s.IndexSchema [dv_coll_shard4_replica_n6] Schema name=minimal
   [junit4]   2> 1137482 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1137482 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'dv_coll_shard4_replica_n6' using configuration from collection dv_coll, trusted=true
   [junit4]   2> 1137483 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33027.solr.core.dv_coll.shard4.replica_n6' (registry 'solr.core.dv_coll.shard4.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446f7595
   [junit4]   2> 1137483 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1137483 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.SolrCore [[dv_coll_shard4_replica_n6] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.DocValuesNotIndexedTest_F2644420F7092010-001/tempDir-001/node1/dv_coll_shard4_replica_n6], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.DocValuesNotIndexedTest_F2644420F7092010-001/tempDir-001/node1/./dv_coll_shard4_replica_n6/data/]
   [junit4]   2> 1137485 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1137492 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.IndexSchema [dv_coll_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 1137497 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.s.IndexSchema [dv_coll_shard3_replica_n4] Schema name=minimal
   [junit4]   2> 1137500 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1137500 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'dv_coll_shard3_replica_n4' using configuration from collection dv_coll, trusted=true
   [junit4]   2> 1137568 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1137568 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'dv_coll_shard1_replica_n1' using configuration from collection dv_coll, trusted=true
   [junit4]   2> 1137568 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33383.solr.core.dv_coll.shard1.replica_n1' (registry 'solr.core.dv_coll.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446f7595
   [junit4]   2> 1137569 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1137569 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.SolrCore [[dv_coll_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.DocValuesNotIndexedTest_F2644420F7092010-001/tempDir-001/node2/dv_coll_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.DocValuesNotIndexedTest_F2644420F7092010-001/tempDir-001/node2/./dv_coll_shard1_replica_n1/data/]
   [junit4]   2> 1137574 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33383.solr.core.dv_coll.shard3.replica_n4' (registry 'solr.core.dv_coll.shard3.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446f7595
   [junit4]   2> 1137574 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1137574 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.SolrCore [[dv_coll_shard3_replica_n4] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.DocValuesNotIndexedTest_F2644420F7092010-001/tempDir-001/node2/dv_coll_shard3_replica_n4], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.DocValuesNotIndexedTest_F2644420F7092010-001/tempDir-001/node2/./dv_coll_shard3_replica_n4/data/]
   [junit4]   2> 1137612 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1137612 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1137614 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1137614 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1137616 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@334af08e[dv_coll_shard2_replica_n2] main]
   [junit4]   2> 1137633 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1137633 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1137634 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1137634 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1137634 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1137635 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 1137635 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1137635 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1137636 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1137636 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590310969835257856
   [junit4]   2> 1137637 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@53bc2d1a[dv_coll_shard4_replica_n6] main]
   [junit4]   2> 1137638 INFO  (searcherExecutor-4492-thread-1-processing-n:127.0.0.1:33027_solr x:dv_coll_shard2_replica_n2 s:shard2 c:dv_coll r:core_node5) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SolrCore [dv_coll_shard2_replica_n2] Registered new searcher Searcher@334af08e[dv_coll_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1137662 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1137662 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1137662 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1137664 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1137664 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1137666 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@59b9ce01[dv_coll_shard3_replica_n4] main]
   [junit4]   2> 1137668 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1137668 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1137670 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1137670 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1137670 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1137670 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1137672 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@bd05087[dv_coll_shard1_replica_n1] main]
   [junit4]   2> 1137674 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 1137674 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1137675 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590310969876152320
   [junit4]   2> 1137677 INFO  (searcherExecutor-4493-thread-1-processing-n:127.0.0.1:33027_solr x:dv_coll_shard4_replica_n6 s:shard4 c:dv_coll r:core_node8) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.SolrCore [dv_coll_shard4_replica_n6] Registered new searcher Searcher@53bc2d1a[dv_coll_shard4_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1137680 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1137680 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1137680 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33027/solr/dv_coll_shard2_replica_n2/
   [junit4]   2> 1137680 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1137680 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:33027/solr/dv_coll_shard2_replica_n2/ has no replicas
   [junit4]   2> 1137680 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1137681 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1137681 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1137682 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1137682 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33027/solr/dv_coll_shard4_replica_n6/
   [junit4]   2> 1137682 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1137682 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.SyncStrategy http://127.0.0.1:33027/solr/dv_coll_shard4_replica_n6/ has no replicas
   [junit4]   2> 1137682 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1137682 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1137683 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33027/solr/dv_coll_shard2_replica_n2/ shard2
   [junit4]   2> 1137684 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1137684 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1137684 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1137684 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1137686 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33027/solr/dv_coll_shard4_replica_n6/ shard4
   [junit4]   2> 1137692 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 1137693 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1137693 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590310969895026688
   [junit4]   2> 1137696 INFO  (searcherExecutor-4495-thread-1-processing-n:127.0.0.1:33383_solr x:dv_coll_shard1_replica_n1 s:shard1 c:dv_coll r:core_node3) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.SolrCore [dv_coll_shard1_replica_n1] Registered new searcher Searcher@bd05087[dv_coll_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1137696 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 1137697 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1137698 INFO  (searcherExecutor-4494-thread-1-processing-n:127.0.0.1:33383_solr x:dv_coll_shard3_replica_n4 s:shard3 c:dv_coll r:core_node7) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.SolrCore [dv_coll_shard3_replica_n4] Registered new searcher Searcher@59b9ce01[dv_coll_shard3_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1137699 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590310969901318144
   [junit4]   2> 1137719 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1137719 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1137719 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33383/solr/dv_coll_shard3_replica_n4/
   [junit4]   2> 1137719 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1137719 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.SyncStrategy http://127.0.0.1:33383/solr/dv_coll_shard3_replica_n4/ has no replicas
   [junit4]   2> 1137719 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1137719 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1137719 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1137720 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33383/solr/dv_coll_shard1_replica_n1/
   [junit4]   2> 1137720 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1137720 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:33383/solr/dv_coll_shard1_replica_n1/ has no replicas
   [junit4]   2> 1137720 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1137722 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33383/solr/dv_coll_shard1_replica_n1/ shard1
   [junit4]   2> 1137722 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33383/solr/dv_coll_shard3_replica_n4/ shard3
   [junit4]   2> 1137838 INFO  (zkCallback-2236-thread-2-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1137838 INFO  (zkCallback-2238-thread-2-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1137838 INFO  (zkCallback-2236-thread-1-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1137838 INFO  (zkCallback-2238-thread-1-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1137878 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1137878 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1137884 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=dv_coll_shard1_replica_n1&action=CREATE&numShards=4&collection=dv_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1450
   [junit4]   2> 1137885 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1137887 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1137887 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=dv_coll_shard2_replica_n2&action=CREATE&numShards=4&collection=dv_coll&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1466
   [junit4]   2> 1137889 INFO  (qtp1396299069-12192) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=dv_coll_shard4_replica_n6&action=CREATE&numShards=4&collection=dv_coll&shard=shard4&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1448
   [junit4]   2> 1137891 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=dv_coll_shard3_replica_n4&action=CREATE&numShards=4&collection=dv_coll&shard=shard3&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1449
   [junit4]   2> 1137894 INFO  (qtp1396299069-12187) [n:127.0.0.1:33027_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1137993 INFO  (zkCallback-2238-thread-2-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1137993 INFO  (zkCallback-2238-thread-1-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1137994 INFO  (zkCallback-2236-thread-2-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1137994 INFO  (zkCallback-2236-thread-1-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1138138 INFO  (OverseerCollectionConfigSetProcessor-99394435498442757-127.0.0.1:33383_solr-n_0000000000) [n:127.0.0.1:33383_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1138894 INFO  (qtp1396299069-12187) [n:127.0.0.1:33027_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&maxShardsPerNode=2&name=dv_coll&nrtReplicas=1&action=CREATE&numShards=4&wt=javabin&version=2} status=0 QTime=2781
   [junit4]   2> 1138901 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.SchemaManager managed schema loaded . version : 0
   [junit4]   2> 1138903 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.IndexSchema [dv_coll_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 1138905 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1138911 INFO  (zkCallback-2236-thread-2-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.s.ZkIndexSchemaReader A schema change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/configs/conf1/managed-schema, has occurred - updating schema from ZooKeeper ...
   [junit4]   2> 1138911 INFO  (zkCallback-2236-thread-3-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.s.ZkIndexSchemaReader A schema change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/configs/conf1/managed-schema, has occurred - updating schema from ZooKeeper ...
   [junit4]   2> 1138911 INFO  (zkCallback-2238-thread-1-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.s.ZkIndexSchemaReader A schema change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/configs/conf1/managed-schema, has occurred - updating schema from ZooKeeper ...
   [junit4]   2> 1138911 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.ZkController Persisted config data to node /configs/conf1/managed-schema
   [junit4]   2> 1138911 INFO  (zkCallback-2238-thread-2-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.s.ZkIndexSchemaReader A schema change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/configs/conf1/managed-schema, has occurred - updating schema from ZooKeeper ...
   [junit4]   2> 1138911 INFO  (zkCallback-2236-thread-2-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 1 from ZooKeeper
   [junit4]   2> 1138911 INFO  (zkCallback-2236-thread-3-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 1 from ZooKeeper
   [junit4]   2> 1138913 INFO  (zkCallback-2236-thread-2-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.s.IndexSchema [dv_coll_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 1138913 INFO  (zkCallback-2236-thread-3-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.s.IndexSchema [dv_coll_shard3_replica_n4] Schema name=minimal
   [junit4]   2> 1138917 INFO  (zkCallback-2236-thread-3-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1138917 INFO  (zkCallback-2236-thread-3-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.s.ZkIndexSchemaReader Finished refreshing schema in 5 ms
   [junit4]   2> 1138917 INFO  (zkCallback-2236-thread-2-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1138917 INFO  (zkCallback-2236-thread-2-processing-n:127.0.0.1:33383_solr) [n:127.0.0.1:33383_solr    ] o.a.s.s.ZkIndexSchemaReader Finished refreshing schema in 5 ms
   [junit4]   2> 1138917 INFO  (zkCallback-2238-thread-2-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 1 from ZooKeeper
   [junit4]   2> 1138917 INFO  (Thread-4326) [n:127.0.0.1:33383_solr    ] o.a.s.c.SolrCore config update listener called for core dv_coll_shard1_replica_n1
   [junit4]   2> 1138919 INFO  (zkCallback-2238-thread-2-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.s.IndexSchema [dv_coll_shard4_replica_n6] Schema name=minimal
   [junit4]   2> 1138919 INFO  (Thread-4326) [n:127.0.0.1:33383_solr    ] o.a.s.c.SolrCore config update listener called for core dv_coll_shard3_replica_n4
   [junit4]   2> 1138923 INFO  (zkCallback-2238-thread-2-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1138923 INFO  (zkCallback-2238-thread-2-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.s.ZkIndexSchemaReader Finished refreshing schema in 6 ms
   [junit4]   2> 1138925 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1138926 INFO  (Thread-4327) [n:127.0.0.1:33027_solr    ] o.a.s.c.SolrCore config update listener called for core dv_coll_shard4_replica_n6
   [junit4]   2> 1138926 INFO  (Thread-4327) [n:127.0.0.1:33027_solr    ] o.a.s.c.SolrCore config update listener called for core dv_coll_shard2_replica_n2
   [junit4]   2> 1138927 INFO  (Thread-4327) [n:127.0.0.1:33027_solr    ] o.a.s.c.SolrCore core reload dv_coll_shard2_replica_n2
   [junit4]   2> 1138933 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.IndexSchema [dv_coll_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 1138936 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1138936 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.CoreContainer Reloading SolrCore 'dv_coll_shard2_replica_n2' using configuration from collection dv_coll
   [junit4]   2> 1138973 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33027.solr.core.dv_coll.shard2.replica_n2' (registry 'solr.core.dv_coll.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446f7595
   [junit4]   2> 1138973 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SolrCore [[dv_coll_shard2_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.DocValuesNotIndexedTest_F2644420F7092010-001/tempDir-001/node1/dv_coll_shard2_replica_n2], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.DocValuesNotIndexedTest_F2644420F7092010-001/tempDir-001/node1/./dv_coll_shard2_replica_n2/data/]
   [junit4]   2> 1139010 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1139010 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1139012 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@7d076fd6[dv_coll_shard2_replica_n2] main]
   [junit4]   2> 1139013 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1139013 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1139013 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1139013 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 1 is already the latest
   [junit4]   2> 1139013 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1139014 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590310971280195584
   [junit4]   2> 1139016 INFO  (searcherExecutor-4512-thread-1-processing-n:127.0.0.1:33027_solr x:dv_coll_shard2_replica_n2 s:shard2 c:dv_coll r:core_node5) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SolrCore [dv_coll_shard2_replica_n2] Registered new searcher Searcher@7d076fd6[dv_coll_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1139017 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1139017 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@e2eb458[dv_coll_shard2_replica_n2] main]
   [junit4]   2> 1139018 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.ManagedIndexSchema Waiting up to 599 secs for 3 replicas to apply schema update version 1 for collection dv_coll
   [junit4]   2> 1139018 INFO  (zkCallback-2238-thread-1-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 1 from ZooKeeper
   [junit4]   2> 1139020 INFO  (searcherExecutor-4512-thread-1-processing-n:127.0.0.1:33027_solr x:dv_coll_shard2_replica_n2 s:shard2 c:dv_coll r:core_node5) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SolrCore [dv_coll_shard2_replica_n2] Registered new searcher Searcher@e2eb458[dv_coll_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1139020 INFO  (zkCallback-2238-thread-1-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.s.IndexSchema [dv_coll_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 1139023 INFO  (Thread-4327) [n:127.0.0.1:33027_solr    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1139023 INFO  (zkCallback-2238-thread-1-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1139024 INFO  (zkCallback-2238-thread-1-processing-n:127.0.0.1:33027_solr) [n:127.0.0.1:33027_solr    ] o.a.s.s.ZkIndexSchemaReader Finished refreshing schema in 5 ms
   [junit4]   2> 1139030 INFO  (Thread-4327) [n:127.0.0.1:33027_solr    ] o.a.s.s.IndexSchema [dv_coll_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 1139034 INFO  (Thread-4327) [n:127.0.0.1:33027_solr    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1139034 INFO  (Thread-4327) [n:127.0.0.1:33027_solr    ] o.a.s.c.CoreContainer Reloading SolrCore 'dv_coll_shard2_replica_n2' using configuration from collection dv_coll
   [junit4]   2> 1139046 INFO  (qtp137640639-12258) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.S.Request [dv_coll_shard3_replica_n4]  webapp=/solr path=/schema/zkversion params={refreshIfBelowVersion=1&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1139046 INFO  (qtp1396299069-12261) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.S.Request [dv_coll_shard4_replica_n6]  webapp=/solr path=/schema/zkversion params={refreshIfBelowVersion=1&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1139057 INFO  (qtp137640639-12258) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.S.Request [dv_coll_shard1_replica_n1]  webapp=/solr path=/schema/zkversion params={refreshIfBelowVersion=1&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1139057 INFO  (Thread-4327) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33027.solr.core.dv_coll.shard2.replica_n2' (registry 'solr.core.dv_coll.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446f7595
   [junit4]   2> 1139057 INFO  (Thread-4327) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SolrCore [[dv_coll_shard2_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.DocValuesNotIndexedTest_F2644420F7092010-001/tempDir-001/node1/dv_coll_shard2_replica_n2], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.DocValuesNotIndexedTest_F2644420F7092010-001/tempDir-001/node1/./dv_coll_shard2_replica_n2/data/]
   [junit4]   2> 1139057 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.ManagedIndexSchema Took 39.0ms for 3 replicas to apply schema update version 1 for collection dv_coll
   [junit4]   2> 1139057 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.S.Request [dv_coll_shard2_replica_n2]  webapp=/solr path=/schema params={wt=javabin&version=2} status=0 QTime=159
   [junit4]   2> 1139057 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SolrCore [dv_coll_shard2_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@7d28fb9c
   [junit4]   2> 1139057 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.dv_coll.shard2.replica_n2, tag=2099837852
   [junit4]   2> 1139057 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@172255fd: rootName = solr_33027, domain = solr.core.dv_coll.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.dv_coll.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@4131bd98
   [junit4]   2> 1139064 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.dv_coll.shard2.leader, tag=2099837852
   [junit4]   2> 1139090 INFO  (TEST-DocValuesNotIndexedTest.testGroupingDVOnly-seed#[F2644420F7092010]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testGroupingDVOnly
   [junit4]   2> 1139114 INFO  (qtp137640639-12258) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard3_replica_n4]  webapp=/solr path=/update params={update.distrib=TOLEADER&distrib.from=http://127.0.0.1:33027/solr/dv_coll_shard2_replica_n2/&wt=javabin&version=2}{deleteByQuery=*:* (-1590310971365130240)} 0 18
   [junit4]   2> 1139116 INFO  (qtp1396299069-12262) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard4_replica_n6]  webapp=/solr path=/update params={update.distrib=TOLEADER&distrib.from=http://127.0.0.1:33027/solr/dv_coll_shard2_replica_n2/&wt=javabin&version=2}{deleteByQuery=*:* (-1590310971365130240)} 0 20
   [junit4]   2> 1139116 INFO  (qtp137640639-12199) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=TOLEADER&distrib.from=http://127.0.0.1:33027/solr/dv_coll_shard2_replica_n2/&wt=javabin&version=2}{deleteByQuery=*:* (-1590310971365130240)} 0 21
   [junit4]   2> 1139117 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard2_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:* (-1590310971364081664)} 0 26
   [junit4]   2> 1139127 INFO  (qtp137640639-12258) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1590310971398684672,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1139127 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1590310971398684672,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1139127 INFO  (qtp137640639-12258) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1139127 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1139127 INFO  (qtp1396299069-12189) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1590310971398684672,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1139127 INFO  (qtp137640639-12258) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@788aebda[dv_coll_shard1_replica_n1] main]
   [junit4]   2> 1139127 INFO  (qtp1396299069-12189) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1139127 INFO  (qtp137640639-12253) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1590310971398684672,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1139127 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1139127 INFO  (qtp137640639-12253) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1139127 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard2_replica_n2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:33383/solr/dv_coll_shard1_replica_n1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
   [junit4]   2> 1139127 INFO  (qtp1396299069-12189) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@39fde5e9[dv_coll_shard4_replica_n6] main]
   [junit4]   2> 1139128 INFO  (qtp137640639-12253) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@42c981c1[dv_coll_shard3_replica_n4] main]
   [junit4]   2> 1139128 INFO  (qtp137640639-12258) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1139128 INFO  (qtp1396299069-12189) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1139128 INFO  (qtp137640639-12253) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1139129 INFO  (searcherExecutor-4494-thread-1-processing-n:127.0.0.1:33383_solr x:dv_coll_shard3_replica_n4 s:shard3 c:dv_coll r:core_node7) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.SolrCore [dv_coll_shard3_replica_n4] Registered new searcher Searcher@42c981c1[dv_coll_shard3_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1139129 INFO  (searcherExecutor-4493-thread-1-processing-n:127.0.0.1:33027_solr x:dv_coll_shard4_replica_n6 s:shard4 c:dv_coll r:core_node8) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.SolrCore [dv_coll_shard4_replica_n6] Registered new searcher Searcher@39fde5e9[dv_coll_shard4_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1139130 INFO  (qtp137640639-12253) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard3_replica_n4]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:33383/solr/dv_coll_shard1_replica_n1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 2
   [junit4]   2> 1139130 INFO  (qtp1396299069-12189) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard4_replica_n6]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:33383/solr/dv_coll_shard1_replica_n1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 2
   [junit4]   2> 1139129 INFO  (searcherExecutor-4495-thread-1-processing-n:127.0.0.1:33383_solr x:dv_coll_shard1_replica_n1 s:shard1 c:dv_coll r:core_node3) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.SolrCore [dv_coll_shard1_replica_n1] Registered new searcher Searcher@788aebda[dv_coll_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1139130 INFO  (qtp137640639-12258) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:33383/solr/dv_coll_shard1_replica_n1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 3
   [junit4]   2> 1139130 INFO  (qtp137640639-12199) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard1_replica_n1]  webapp=/solr path=/update params={_stateVer_=dv_coll:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 11
   [junit4]   2> 1139131 INFO  (Thread-4327) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1139131 INFO  (Thread-4327) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1139132 INFO  (Thread-4327) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@4e6be2a9[dv_coll_shard2_replica_n2] main]
   [junit4]   2> 1139133 INFO  (Thread-4327) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1139135 INFO  (Thread-4327) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1139135 INFO  (Thread-4327) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1139136 INFO  (Thread-4327) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 1 is already the latest
   [junit4]   2> 1139136 INFO  (Thread-4327) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1139139 INFO  (qtp1396299069-12196) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard4_replica_n6]  webapp=/solr path=/update params={_stateVer_=dv_coll:4&wt=javabin&version=2}{add=[10000 (1590310971404976128), 5 (1590310971408121856), 17 (1590310971409170432), 19 (1590310971410219008), 21 (1590310971410219009), 23 (1590310971410219010), 25 (1590310971410219011), 29 (1590310971410219012), 10030 (1590310971410219013), 33 (1590310971410219014), ... (13 adds)]} 0 5
   [junit4]   2> 1139139 INFO  (qtp137640639-12191) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard3_replica_n4]  webapp=/solr path=/update params={_stateVer_=dv_coll:4&wt=javabin&version=2}{add=[2 (1590310971404976128), 3 (1590310971410219008), 6 (1590310971410219009), 7 (1590310971410219010), 9 (1590310971410219011), 10010 (1590310971411267584), 18 (1590310971411267585), 10025 (1590310971411267586), 26 (1590310971411267587), 30 (1590310971411267588), ... (14 adds)]} 0 6
   [junit4]   2> 1139141 INFO  (searcherExecutor-4517-thread-1-processing-n:127.0.0.1:33027_solr x:dv_coll_shard2_replica_n2 s:shard2 c:dv_coll r:core_node5) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SolrCore [dv_coll_shard2_replica_n2] Registered new searcher Searcher@4e6be2a9[dv_coll_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1139142 INFO  (qtp137640639-12258) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard1_replica_n1]  webapp=/solr path=/update params={_stateVer_=dv_coll:4&wt=javabin&version=2}{add=[1 (1590310971408121856), 8 (1590310971413364736), 10 (1590310971413364737), 11 (1590310971413364738), 13 (1590310971413364739), 14 (1590310971413364740), 10015 (1590310971413364741), 10020 (1590310971413364742), 24 (1590310971413364743), 27 (1590310971414413312), ... (16 adds)]} 0 6
   [junit4]   2> 1139145 INFO  (Thread-4327) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1139147 INFO  (qtp1396299069-12200) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [dv_coll_shard2_replica_n2]  webapp=/solr path=/update params={_stateVer_=dv_coll:4&wt=javabin&version=2}{add=[0 (1590310971408121856), 4 (1590310971412316160), 10005 (1590310971413364736), 12 (1590310971418607616), 15 (1590310971418607617), 16 (1590310971418607618), 20 (1590310971418607619), 22 (1590310971418607620), 34 (1590310971418607621), 35 (1590310971419656192), ... (16 adds)]} 0 11
   [junit4]   2> 1139149 INFO  (qtp1396299069-12198) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1590310971421753344,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1139149 INFO  (qtp1396299069-12262) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1590310971421753344,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1139149 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1590310971421753344,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1139149 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1590310971421753344,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1139150 INFO  (qtp1396299069-12262) [n:127.0.0.1:33027_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@5b0041fd commitCommandVersion:1590310971421753344
   [junit4]   2> 1139150 INFO  (qtp1396299069-12198) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@b5f7bc4 commitCommandVersion:1590310971421753344
   [junit4]   2> 1139151 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@5c63bbe commitCommandVersion:1590310971421753344
   [junit4]   2> 1139151 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1c5c85d2 commitCommandVersion:1590310971421753344
   [junit4]   2> 1139158 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4f4c7338[dv_coll_shard1_replica_n1] main]
   [junit4]   2> 1139159 INFO  (qtp137640639-12190) [n:127.0.0.1:33383_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1139163 INFO  (qtp1396299069-12198) [n:127.0.0.1:33027_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@7b0c0d6e[dv_coll_shard4_replica_n6] main]
   [junit4]   2> 1139164 INFO  (qtp137640639-12188) [n:127.0.0.1:33383_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_repl

[...truncated too long message...]

2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1280953 INFO  (coreCloseExecutor-4418-thread-1) [n:127.0.0.1:37309_solr c:.system s:shard1 r:core_node8 x:.system_shard1_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection..system.shard1.leader, tag=1961402838
   [junit4]   2> 1280954 INFO  (coreCloseExecutor-4418-thread-2) [n:127.0.0.1:37309_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node8 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.AutoscalingHistoryHandlerTest_collection.shard1.leader, tag=441597580
   [junit4]   2> 1280954 INFO  (zkCallback-2196-thread-5-processing-n:127.0.0.1:37309_solr) [n:127.0.0.1:37309_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37309_solr
   [junit4]   2> 1280955 INFO  (jetty-closer-2151-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6f4dd5bb{/solr,null,UNAVAILABLE}
   [junit4]   2> 1280955 INFO  (jetty-closer-2151-thread-2) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 1280956 INFO  (jetty-closer-2151-thread-3) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@55ad544f{/solr,null,UNAVAILABLE}
   [junit4]   2> 1280956 INFO  (jetty-closer-2151-thread-3) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 1282453 INFO  (jetty-closer-2151-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3de34175{/solr,null,UNAVAILABLE}
   [junit4]   2> 1282454 INFO  (jetty-closer-2151-thread-1) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 1282454 ERROR (SUITE-AutoscalingHistoryHandlerTest-seed#[F2644420F7092010]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1282454 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[F2644420F7092010]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33831 33831
   [junit4]   2> 1287559 INFO  (Thread-2873) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33831 33831
   [junit4]   2> 1287559 WARN  (Thread-2873) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 13 /solr/aliases.json
   [junit4]   2> 5 /solr/configs/.system
   [junit4]   2> 5 /solr/configs/.system/managed-schema
   [junit4]   2> 4 /solr/security.json
   [junit4]   2> 4 /solr/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/managed-schema
   [junit4]   2> 4 /solr/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 56 /solr/collections/.system/state.json
   [junit4]   2> 40 /solr/collections/AutoscalingHistoryHandlerTest_collection/state.json
   [junit4]   2> 13 /solr/clusterprops.json
   [junit4]   2> 13 /solr/clusterstate.json
   [junit4]   2> 2 /solr/collections/.system/leader_elect/shard1/election/99394442677714952-core_node3-n_0000000001
   [junit4]   2> 2 /solr/autoscaling.json
   [junit4]   2> 2 /solr/collections/AutoscalingHistoryHandlerTest_collection/leader_elect/shard1/election/99394442677714952-core_node5-n_0000000000
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 13 /solr/live_nodes
   [junit4]   2> 13 /solr/collections
   [junit4]   2> 2 /solr/overseer/queue
   [junit4]   2> 2 /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 2 /solr/autoscaling/events/AutoscalingHistoryHandlerTest_node_lost_trigger
   [junit4]   2> 2 /solr/overseer/collection-queue-work
   [junit4]   2> 2 /solr/overseer/queue-work
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F2644420F7092010-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {action_s=PostingsFormat(name=Asserting), event.source_s=PostingsFormat(name=Direct), type=PostingsFormat(name=Asserting), before.actions_ss=PostingsFormat(name=Direct), event.property._enqueue_time__ss=PostingsFormat(name=Asserting), after.actions_ss=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), stage_s=PostingsFormat(name=Asserting), event.id_s=BlockTreeOrds(blocksize=128), event.property.eventTimes_ss=PostingsFormat(name=Asserting), source_s=BlockTreeOrds(blocksize=128), event.type_s=PostingsFormat(name=Asserting), responses_ts=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), id=BlockTreeOrds(blocksize=128), collections_ss=PostingsFormat(name=Asserting), operations.params_ts=PostingsFormat(name=Asserting), event.property.nodeNames_ss=PostingsFormat(name=Asserting)}, docValues:{action_s=DocValuesFormat(name=Direct), context_str=DocValuesFormat(name=Lucene70), event_str=DocValuesFormat(name=Asserting), event.source_s=DocValuesFormat(name=Lucene70), type=DocValuesFormat(name=Direct), event.property._enqueue_time__ss=DocValuesFormat(name=Direct), before.actions_ss=DocValuesFormat(name=Lucene70), after.actions_ss=DocValuesFormat(name=Asserting), stage_s=DocValuesFormat(name=Direct), _version_=DocValuesFormat(name=Direct), event.id_s=DocValuesFormat(name=Lucene70), event.property.eventTimes_ss=DocValuesFormat(name=Direct), source_s=DocValuesFormat(name=Lucene70), event.type_s=DocValuesFormat(name=Direct), responses_ts=DocValuesFormat(name=Asserting), event.time_l=DocValuesFormat(name=Direct), id=DocValuesFormat(name=Lucene70), operations.params_ts=DocValuesFormat(name=Direct), collections_ss=DocValuesFormat(name=Direct), event.property.nodeNames_ss=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=1989, maxMBSortInHeap=5.719383594035442, sim=RandomSimilarity(queryNorm=true): {}, locale=fi, timezone=America/Argentina/Cordoba
   [junit4]   2> NOTE: Linux 4.13.0-26-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=8,threads=1,free=116667816,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [FileUtilsTest, TestUtilizeNode, TestBM25SimilarityFactory, PrimUtilsTest, LoggingHandlerTest, TestCloudInspectUtil, RulesTest, SearchHandlerTest, TestJavabinTupleStreamParser, OverseerTest, HLLUtilTest, TermsComponentTest, CurrencyFieldTypeTest, ConnectionManagerTest, HdfsCollectionsAPIDistributedZkTest, CdcrRequestHandlerTest, TestComputePlanAction, ComputePlanActionTest, MoveReplicaHDFSFailoverTest, SOLR749Test, TestAddFieldRealTimeGet, TestTriggerIntegration, FieldMutatingUpdateProcessorTest, TestDFISimilarityFactory, TestReload, BigEndianAscendingWordDeserializerTest, TestLegacyField, TestComplexPhraseQParserPlugin, AsyncCallRequestStatusResponseTest, SolrInfoBeanTest, TestNumericRangeQuery64, SolrXmlInZkTest, TestSweetSpotSimilarityFactory, TestLRUCache, BadCopyFieldTest, PeerSyncReplicationTest, ExplicitHLLTest, SyncSliceTest, TestImplicitCoreProperties, TestTlogReplica, XsltUpdateRequestHandlerTest, TestSchemaVersionResource, TestExactSharedStatsCache, TestFieldCacheVsDocValues, TestFunctionQuery, QueryParsingTest, TestPerFieldSimilarity, TestRestManager, TestNumericTerms32, AutoAddReplicasIntegrationTest, QueryResultKeyTest, DeleteLastCustomShardedReplicaTest, DataDrivenBlockJoinTest, SystemInfoHandlerTest, DOMUtilTest, ChaosMonkeySafeLeaderTest, HttpTriggerListenerTest, TestInPlaceUpdatesDistrib, GraphQueryTest, TestQueryWrapperFilter, ScriptEngineTest, TestObjectReleaseTracker, TestTestInjection, BlockDirectoryTest, CollectionsAPIDistributedZkTest, AtomicUpdatesTest, TestSolrJ, ZkControllerTest, TestEmbeddedSolrServerAdminHandler, TestSearchPerf, TestReqParamsAPI, AlternateDirectoryTest, TestFoldingMultitermQuery, TimeRoutedAliasUpdateProcessorTest, TestUseDocValuesAsStored2, LegacyCloudClusterPropTest, RollingRestartTest, BasicAuthIntegrationTest, BasicFunctionalityTest, TestDistributedStatsComponentCardinality, TestSolrCloudWithDelegationTokens, HdfsRestartWhileUpdatingTest, SignatureUpdateProcessorFactoryTest, BasicZkTest, CollectionReloadTest, TestSurroundQueryParser, QueryElevationComponentTest, SchemaVersionSpecificBehaviorTest, ReplaceNodeTest, TestSimpleTextCodec, BadComponentTest, TestSchemaManager, HdfsNNFailoverTest, TestExportWriter, TestReplicationHandler, TestFastLRUCache, TestLeaderElectionWithEmptyReplica, TestJettySolrRunner, HdfsLockFactoryTest, TestPostingsSolrHighlighter, StatsReloadRaceTest, TestManagedSchemaAPI, AutoscalingHistoryHandlerTest]
   [junit4] Completed [336/767 (2!)] on J1 in 42.39s, 1 test, 1 failure <<< FAILURES!

[...truncated 1429 lines...]
   [junit4] JVM J1: stdout was not empty, see: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/temp/junit4-J1-20180122_161307_9268332566178779206077.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /home/jenkins/workspace/Lucene-Solr-7.x-Linux/heapdumps/java_pid3867.hprof ...
   [junit4] Heap dump file created [344224185 bytes in 0.816 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 8600 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:836: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:788: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid3867.hprof

Total time: 81 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-7.x-Linux (32bit/jdk1.8.0_144) - Build # 1223 - Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1223/
Java: 32bit/jdk1.8.0_144 -client -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey

Error Message:
There are still nodes recoverying - waited for 330 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
        at __randomizedtesting.SeedInfo.seed([E0877BC0E61808C1:6BA0A811A71EA345]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:185)
        at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:140)
        at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:135)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:915)
        at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:431)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13269 lines...]
   [junit4] Suite: org.apache.solr.cloud.api.collections.ShardSplitTest
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/init-core-data-001
   [junit4]   2> 1509303 WARN  (SUITE-ShardSplitTest-seed#[E0877BC0E61808C1]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
   [junit4]   2> 1509303 INFO  (SUITE-ShardSplitTest-seed#[E0877BC0E61808C1]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1509305 INFO  (SUITE-ShardSplitTest-seed#[E0877BC0E61808C1]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 1509305 INFO  (SUITE-ShardSplitTest-seed#[E0877BC0E61808C1]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1509305 INFO  (SUITE-ShardSplitTest-seed#[E0877BC0E61808C1]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1509307 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1509307 INFO  (Thread-4891) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1509307 INFO  (Thread-4891) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1509309 ERROR (Thread-4891) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1509407 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.ZkTestServer start zk server on port:32839
   [junit4]   2> 1509409 INFO  (zkConnectionManagerCallback-1694-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1509413 INFO  (zkConnectionManagerCallback-1696-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1509417 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1509419 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 1509420 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1509421 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1509422 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1509423 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1509424 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1509424 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1509425 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1509426 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1509427 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1509428 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 1509507 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T19:27:37-02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1509508 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1509508 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1509508 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1509508 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@37b238{/,null,AVAILABLE}
   [junit4]   2> 1509510 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@327c7a{HTTP/1.1,[http/1.1]}{127.0.0.1:42677}
   [junit4]   2> 1509510 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.Server Started @1511091ms
   [junit4]   2> 1509510 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/tempDir-001/control/data, hostContext=/, hostPort=42677, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/control-001/cores}
   [junit4]   2> 1509510 ERROR (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1509511 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.3.0
   [junit4]   2> 1509511 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1509511 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1509511 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-22T20:32:41.511Z
   [junit4]   2> 1509512 INFO  (zkConnectionManagerCallback-1698-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1509513 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1509513 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/control-001/solr.xml
   [junit4]   2> 1509517 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1509517 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1509519 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1509523 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32839/solr
   [junit4]   2> 1509524 INFO  (zkConnectionManagerCallback-1702-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1509527 INFO  (zkConnectionManagerCallback-1704-thread-1-processing-n:127.0.0.1:42677_) [n:127.0.0.1:42677_    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1509580 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:42677_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1509581 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:42677_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42677_
   [junit4]   2> 1509581 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:42677_    ] o.a.s.c.Overseer Overseer (id=99395381579481092-127.0.0.1:42677_-n_0000000000) starting
   [junit4]   2> 1509586 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:42677_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42677_
   [junit4]   2> 1509587 INFO  (OverseerStateUpdate-99395381579481092-127.0.0.1:42677_-n_0000000000) [n:127.0.0.1:42677_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1509684 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:42677_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79
   [junit4]   2> 1509694 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:42677_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79
   [junit4]   2> 1509694 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:42677_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79
   [junit4]   2> 1509695 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:42677_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/control-001/cores
   [junit4]   2> 1509715 INFO  (zkConnectionManagerCallback-1710-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1509716 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1509717 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:32839/solr ready
   [junit4]   2> 1509719 INFO  (qtp10910141-11281) [n:127.0.0.1:42677_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:42677_&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1509721 INFO  (OverseerThreadFactory-3400-thread-1-processing-n:127.0.0.1:42677_) [n:127.0.0.1:42677_    ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 1509826 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1509827 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1509934 INFO  (zkCallback-1703-thread-1-processing-n:127.0.0.1:42677_) [n:127.0.0.1:42677_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1510873 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1510896 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 1510993 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 1511004 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 1511004 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79
   [junit4]   2> 1511004 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1511004 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 1511007 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=65.0009765625, floorSegmentMB=0.802734375, forceMergeDeletesPctAllowed=3.1990413320037403, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1511028 WARN  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 1511067 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1511067 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1511069 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1511069 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1511070 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=20, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=95.7275390625, floorSegmentMB=1.806640625, forceMergeDeletesPctAllowed=0.9142756554033027, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.829125549581062
   [junit4]   2> 1511070 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@15e5a1b[control_collection_shard1_replica_n1] main]
   [junit4]   2> 1511072 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1511072 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1511073 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1511074 INFO  (searcherExecutor-3403-thread-1-processing-n:127.0.0.1:42677_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@15e5a1b[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1511074 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590326107123482624
   [junit4]   2> 1511079 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1511079 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1511079 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42677/control_collection_shard1_replica_n1/
   [junit4]   2> 1511079 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1511079 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:42677/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 1511079 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1511082 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42677/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 1511183 INFO  (zkCallback-1703-thread-1-processing-n:127.0.0.1:42677_) [n:127.0.0.1:42677_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1511232 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1511236 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1409
   [junit4]   2> 1511241 INFO  (qtp10910141-11281) [n:127.0.0.1:42677_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1511337 INFO  (zkCallback-1703-thread-1-processing-n:127.0.0.1:42677_) [n:127.0.0.1:42677_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1511724 INFO  (OverseerCollectionConfigSetProcessor-99395381579481092-127.0.0.1:42677_-n_0000000000) [n:127.0.0.1:42677_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1512241 INFO  (qtp10910141-11281) [n:127.0.0.1:42677_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:42677_&wt=javabin&version=2} status=0 QTime=2522
   [junit4]   2> 1512246 INFO  (zkConnectionManagerCallback-1714-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1512248 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1512249 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:32839/solr ready
   [junit4]   2> 1512249 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1512250 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1512252 INFO  (OverseerThreadFactory-3400-thread-2-processing-n:127.0.0.1:42677_) [n:127.0.0.1:42677_    ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 1512252 WARN  (OverseerThreadFactory-3400-thread-2-processing-n:127.0.0.1:42677_) [n:127.0.0.1:42677_    ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 1512458 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1512459 INFO  (qtp10910141-11287) [n:127.0.0.1:42677_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=208
   [junit4]   2> 1512554 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-1-001 of type NRT
   [junit4]   2> 1512555 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T19:27:37-02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1512555 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1512555 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1512555 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1512556 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@75ff4f{/,null,AVAILABLE}
   [junit4]   2> 1512556 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@222ccd{HTTP/1.1,[http/1.1]}{127.0.0.1:44007}
   [junit4]   2> 1512556 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.Server Started @1514138ms
   [junit4]   2> 1512556 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=44007, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-1-001/cores}
   [junit4]   2> 1512557 ERROR (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1512557 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.3.0
   [junit4]   2> 1512557 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1512557 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1512557 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-22T20:32:44.557Z
   [junit4]   2> 1512558 INFO  (zkConnectionManagerCallback-1716-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1512559 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1512559 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-1-001/solr.xml
   [junit4]   2> 1512568 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1512568 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1512570 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1512574 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32839/solr
   [junit4]   2> 1512575 INFO  (zkConnectionManagerCallback-1720-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1512578 INFO  (zkConnectionManagerCallback-1722-thread-1-processing-n:127.0.0.1:44007_) [n:127.0.0.1:44007_    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1512582 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:44007_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1512583 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:44007_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1512584 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:44007_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44007_
   [junit4]   2> 1512585 INFO  (zkCallback-1703-thread-1-processing-n:127.0.0.1:42677_) [n:127.0.0.1:42677_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1512585 INFO  (zkCallback-1713-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1512586 INFO  (zkCallback-1721-thread-1-processing-n:127.0.0.1:44007_) [n:127.0.0.1:44007_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1512785 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:44007_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79
   [junit4]   2> 1512801 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:44007_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79
   [junit4]   2> 1512802 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:44007_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79
   [junit4]   2> 1512804 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:44007_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-1-001/cores
   [junit4]   2> 1512833 INFO  (qtp10910141-11316) [n:127.0.0.1:42677_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:44007_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1512835 INFO  (OverseerCollectionConfigSetProcessor-99395381579481092-127.0.0.1:42677_-n_0000000000) [n:127.0.0.1:42677_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1512836 INFO  (OverseerThreadFactory-3400-thread-3-processing-n:127.0.0.1:42677_) [n:127.0.0.1:42677_    ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:44007_ for creating new replica
   [junit4]   2> 1512839 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1512840 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1512944 INFO  (zkCallback-1721-thread-1-processing-n:127.0.0.1:44007_) [n:127.0.0.1:44007_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1513906 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1513938 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema [collection1_shard2_replica_n41] Schema name=test
   [junit4]   2> 1514040 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 1514075 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n41' using configuration from collection collection1, trusted=true
   [junit4]   2> 1514076 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n41' (registry 'solr.core.collection1.shard2.replica_n41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79
   [junit4]   2> 1514076 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1514076 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [[collection1_shard2_replica_n41] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-1-001/cores/collection1_shard2_replica_n41], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-1-001/cores/collection1_shard2_replica_n41/data/]
   [junit4]   2> 1514081 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=65.0009765625, floorSegmentMB=0.802734375, forceMergeDeletesPctAllowed=3.1990413320037403, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1514104 WARN  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 1514181 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1514181 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1514183 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1514183 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1514186 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=20, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=95.7275390625, floorSegmentMB=1.806640625, forceMergeDeletesPctAllowed=0.9142756554033027, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.829125549581062
   [junit4]   2> 1514187 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.SolrIndexSearcher Opening [Searcher@643784[collection1_shard2_replica_n41] main]
   [junit4]   2> 1514188 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1514189 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1514191 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1514194 INFO  (searcherExecutor-3414-thread-1-processing-n:127.0.0.1:44007_ x:collection1_shard2_replica_n41 s:shard2 c:collection1) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [collection1_shard2_replica_n41] Registered new searcher Searcher@643784[collection1_shard2_replica_n41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1514195 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590326110396088320
   [junit4]   2> 1514204 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1514204 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1514204 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44007/collection1_shard2_replica_n41/
   [junit4]   2> 1514204 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1514204 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy http://127.0.0.1:44007/collection1_shard2_replica_n41/ has no replicas
   [junit4]   2> 1514204 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1514208 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44007/collection1_shard2_replica_n41/ shard2
   [junit4]   2> 1514310 INFO  (zkCallback-1721-thread-1-processing-n:127.0.0.1:44007_) [n:127.0.0.1:44007_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1514359 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1514361 INFO  (qtp20943240-11332) [n:127.0.0.1:44007_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1522
   [junit4]   2> 1514364 INFO  (qtp10910141-11316) [n:127.0.0.1:42677_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:44007_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1530
   [junit4]   2> 1514445 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-2-001 of type NRT
   [junit4]   2> 1514445 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T19:27:37-02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1514446 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1514446 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1514446 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1514446 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1a29bd1{/,null,AVAILABLE}
   [junit4]   2> 1514447 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1a4d861{HTTP/1.1,[http/1.1]}{127.0.0.1:33299}
   [junit4]   2> 1514447 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.Server Started @1516028ms
   [junit4]   2> 1514447 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=33299, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-2-001/cores}
   [junit4]   2> 1514447 ERROR (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1514448 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.3.0
   [junit4]   2> 1514448 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1514448 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1514448 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-22T20:32:46.448Z
   [junit4]   2> 1514450 INFO  (zkConnectionManagerCallback-1726-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1514450 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1514450 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-2-001/solr.xml
   [junit4]   2> 1514456 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1514456 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1514457 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1514460 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32839/solr
   [junit4]   2> 1514462 INFO  (zkConnectionManagerCallback-1730-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1514462 INFO  (zkCallback-1721-thread-1-processing-n:127.0.0.1:44007_) [n:127.0.0.1:44007_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1514464 INFO  (zkConnectionManagerCallback-1732-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1514468 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:33299_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1514469 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:33299_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1514471 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:33299_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33299_
   [junit4]   2> 1514472 INFO  (zkCallback-1721-thread-1-processing-n:127.0.0.1:44007_) [n:127.0.0.1:44007_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1514472 INFO  (zkCallback-1703-thread-1-processing-n:127.0.0.1:42677_) [n:127.0.0.1:42677_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1514472 INFO  (zkCallback-1713-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1514472 INFO  (zkCallback-1731-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1514593 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:33299_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79
   [junit4]   2> 1514607 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:33299_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79
   [junit4]   2> 1514607 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:33299_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79
   [junit4]   2> 1514608 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:33299_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-2-001/cores
   [junit4]   2> 1514664 INFO  (qtp9326367-11364) [n:127.0.0.1:33299_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:33299_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1514666 INFO  (OverseerCollectionConfigSetProcessor-99395381579481092-127.0.0.1:42677_-n_0000000000) [n:127.0.0.1:42677_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1514667 INFO  (OverseerThreadFactory-3400-thread-4-processing-n:127.0.0.1:42677_) [n:127.0.0.1:42677_    ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:33299_ for creating new replica
   [junit4]   2> 1514669 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1514670 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1514774 INFO  (zkCallback-1731-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1514774 INFO  (zkCallback-1721-thread-1-processing-n:127.0.0.1:44007_) [n:127.0.0.1:44007_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1515725 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1515747 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema [collection1_shard1_replica_n43] Schema name=test
   [junit4]   2> 1515844 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 1515864 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n43' using configuration from collection collection1, trusted=true
   [junit4]   2> 1515864 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n43' (registry 'solr.core.collection1.shard1.replica_n43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79
   [junit4]   2> 1515864 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1515865 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [[collection1_shard1_replica_n43] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-2-001/cores/collection1_shard1_replica_n43], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-2-001/cores/collection1_shard1_replica_n43/data/]
   [junit4]   2> 1515869 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=65.0009765625, floorSegmentMB=0.802734375, forceMergeDeletesPctAllowed=3.1990413320037403, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1515891 WARN  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 1515936 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1515936 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1515938 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1515938 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1515940 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=20, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=95.7275390625, floorSegmentMB=1.806640625, forceMergeDeletesPctAllowed=0.9142756554033027, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.829125549581062
   [junit4]   2> 1515940 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.SolrIndexSearcher Opening [Searcher@11a5de4[collection1_shard1_replica_n43] main]
   [junit4]   2> 1515942 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1515942 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1515943 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1515945 INFO  (searcherExecutor-3425-thread-1-processing-n:127.0.0.1:33299_ x:collection1_shard1_replica_n43 s:shard1 c:collection1) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [collection1_shard1_replica_n43] Registered new searcher Searcher@11a5de4[collection1_shard1_replica_n43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1515945 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590326112231096320
   [junit4]   2> 1515949 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1515949 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1515949 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33299/collection1_shard1_replica_n43/
   [junit4]   2> 1515949 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1515949 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy http://127.0.0.1:33299/collection1_shard1_replica_n43/ has no replicas
   [junit4]   2> 1515949 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1515953 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33299/collection1_shard1_replica_n43/ shard1
   [junit4]   2> 1516055 INFO  (zkCallback-1731-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1516055 INFO  (zkCallback-1721-thread-1-processing-n:127.0.0.1:44007_) [n:127.0.0.1:44007_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1516104 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1516106 INFO  (qtp9326367-11370) [n:127.0.0.1:33299_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1437
   [junit4]   2> 1516109 INFO  (qtp9326367-11364) [n:127.0.0.1:33299_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:33299_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1445
   [junit4]   2> 1516193 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-3-001 of type NRT
   [junit4]   2> 1516194 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T19:27:37-02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1516195 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1516195 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1516195 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1516195 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3caf39{/,null,AVAILABLE}
   [junit4]   2> 1516196 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@f0ec8f{HTTP/1.1,[http/1.1]}{127.0.0.1:43351}
   [junit4]   2> 1516196 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.Server Started @1517777ms
   [junit4]   2> 1516196 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/tempDir-001/jetty3, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=43351, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-3-001/cores}
   [junit4]   2> 1516197 ERROR (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1516197 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.3.0
   [junit4]   2> 1516198 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1516198 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1516198 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-22T20:32:48.198Z
   [junit4]   2> 1516201 INFO  (zkConnectionManagerCallback-1736-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1516202 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1516202 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-3-001/solr.xml
   [junit4]   2> 1516207 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1516207 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1516207 INFO  (zkCallback-1731-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1516207 INFO  (zkCallback-1721-thread-1-processing-n:127.0.0.1:44007_) [n:127.0.0.1:44007_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1516208 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1516211 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32839/solr
   [junit4]   2> 1516212 INFO  (zkConnectionManagerCallback-1740-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1516216 INFO  (zkConnectionManagerCallback-1742-thread-1-processing-n:127.0.0.1:43351_) [n:127.0.0.1:43351_    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1516222 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:43351_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1516224 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:43351_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1516225 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:43351_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43351_
   [junit4]   2> 1516226 INFO  (zkCallback-1703-thread-1-processing-n:127.0.0.1:42677_) [n:127.0.0.1:42677_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1516226 INFO  (zkCallback-1721-thread-1-processing-n:127.0.0.1:44007_) [n:127.0.0.1:44007_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1516228 INFO  (zkCallback-1713-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1516228 INFO  (zkCallback-1741-thread-1-processing-n:127.0.0.1:43351_) [n:127.0.0.1:43351_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1516228 INFO  (zkCallback-1731-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1516649 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:43351_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79
   [junit4]   2> 1516661 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:43351_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79
   [junit4]   2> 1516661 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:43351_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79
   [junit4]   2> 1516663 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [n:127.0.0.1:43351_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-3-001/cores
   [junit4]   2> 1516667 INFO  (OverseerCollectionConfigSetProcessor-99395381579481092-127.0.0.1:42677_-n_0000000000) [n:127.0.0.1:42677_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1516713 INFO  (qtp9326367-11368) [n:127.0.0.1:33299_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:43351_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1516716 INFO  (OverseerThreadFactory-3400-thread-5-processing-n:127.0.0.1:42677_) [n:127.0.0.1:42677_    ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:43351_ for creating new replica
   [junit4]   2> 1516719 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1516719 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1516824 INFO  (zkCallback-1731-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1516824 INFO  (zkCallback-1741-thread-1-processing-n:127.0.0.1:43351_) [n:127.0.0.1:43351_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1516824 INFO  (zkCallback-1721-thread-1-processing-n:127.0.0.1:44007_) [n:127.0.0.1:44007_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1517786 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1517813 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.IndexSchema [collection1_shard2_replica_n45] Schema name=test
   [junit4]   2> 1517891 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 1517905 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n45' using configuration from collection collection1, trusted=true
   [junit4]   2> 1517906 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n45' (registry 'solr.core.collection1.shard2.replica_n45') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@bcc79
   [junit4]   2> 1517906 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1517906 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrCore [[collection1_shard2_replica_n45] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-3-001/cores/collection1_shard2_replica_n45], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-3-001/cores/collection1_shard2_replica_n45/data/]
   [junit4]   2> 1517909 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=65.0009765625, floorSegmentMB=0.802734375, forceMergeDeletesPctAllowed=3.1990413320037403, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1517931 WARN  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 1517981 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1517981 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1517982 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1517982 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1517984 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=20, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=95.7275390625, floorSegmentMB=1.806640625, forceMergeDeletesPctAllowed=0.9142756554033027, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.829125549581062
   [junit4]   2> 1517984 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.SolrIndexSearcher Opening [Searcher@6b95bc[collection1_shard2_replica_n45] main]
   [junit4]   2> 1517985 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1517986 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1517986 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1517986 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590326114371239936
   [junit4]   2> 1517988 INFO  (searcherExecutor-3436-thread-1-processing-n:127.0.0.1:43351_ x:collection1_shard2_replica_n45 s:shard2 c:collection1) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrCore [collection1_shard2_replica_n45] Registered new searcher Searcher@6b95bc[collection1_shard2_replica_n45] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1517989 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.ZkController Core needs to recover:collection1_shard2_replica_n45
   [junit4]   2> 1517989 INFO  (updateExecutor-1737-thread-1-processing-n:127.0.0.1:43351_ x:collection1_shard2_replica_n45 s:shard2 c:collection1) [n:127.0.0.1:43351_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 1517989 INFO  (recoveryExecutor-1738-thread-1-processing-n:127.0.0.1:43351_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:43351_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 1517990 INFO  (recoveryExecutor-1738-thread-1-processing-n:127.0.0.1:43351_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:43351_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 1517990 INFO  (recoveryExecutor-1738-thread-1-processing-n:127.0.0.1:43351_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:43351_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard2_replica_n45]
   [junit4]   2> 1517990 INFO  (recoveryExecutor-1738-thread-1-processing-n:127.0.0.1:43351_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:43351_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 1517990 INFO  (recoveryExecutor-1738-thread-1-processing-n:127.0.0.1:43351_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:43351_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard2_replica_n45] as recovering, leader is [http://127.0.0.1:44007/collection1_shard2_replica_n41/] and I am [http://127.0.0.1:43351/collection1_shard2_replica_n45/]
   [junit4]   2> 1517990 INFO  (qtp32199626-11397) [n:127.0.0.1:43351_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1271
   [junit4]   2> 1517993 INFO  (qtp9326367-11368) [n:127.0.0.1:33299_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:43351_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1279
   [junit4]   2> 1517993 INFO  (recoveryExecutor-1738-thread-1-processing-n:127.0.0.1:43351_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:43351_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:44007]; [WaitForState: action=PREPRECOVERY&core=collection1_shard2_replica_n41&nodeName=127.0.0.1:43351_&coreNodeName=core_node46&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 1517994 INFO  (qtp20943240-11338) [n:127.0.0.1:44007_    ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node46, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
   [junit4]   2> 1517994 INFO  (qtp20943240-11338) [n:127.0.0.1:44007_    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1_shard2_replica_n41, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:43351_, coreNodeName=core_node46, onlyIfActiveCheckResult=false, nodeProps: core_node46:{"core":"collection1_shard2_replica_n45","base_url":"http://127.0.0.1:43351","node_name":"127.0.0.1:43351_","state":"down","type":"NRT"}
   [junit4]   2> 1518063 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-4-001 of type NRT
   [junit4]   2> 1518064 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T19:27:37-02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1518065 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1518065 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1518065 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1518065 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5b4f4a{/,null,AVAILABLE}
   [junit4]   2> 1518066 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7207cb{HTTP/1.1,[http/1.1]}{127.0.0.1:41173}
   [junit4]   2> 1518066 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.Server Started @1519647ms
   [junit4]   2> 1518066 INFO  (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/tempDir-001/jetty4, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=41173, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001/shard-4-001/cores}
   [junit4]   2> 1518066 ERROR (TEST

[...truncated too long message...]

utor-4136-thread-3) [n:127.0.0.1:40525_ c:splitByRouteKeyTest s:shard4 r:core_node16 x:splitByRouteKeyTest_shard4_replica_n14] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.splitByRouteKeyTest.shard4.leader, tag=14615488
   [junit4]   2> 2436582 INFO  (coreCloseExecutor-4136-thread-4) [n:127.0.0.1:40525_ c:splitByRouteKeyTest s:shard2 r:core_node7 x:splitByRouteKeyTest_shard2_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.splitByRouteKeyTest.shard2.replica_n4, tag=15057616
   [junit4]   2> 2436582 INFO  (coreCloseExecutor-4136-thread-4) [n:127.0.0.1:40525_ c:splitByRouteKeyTest s:shard2 r:core_node7 x:splitByRouteKeyTest_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@9b00ff: rootName = null, domain = solr.core.splitByRouteKeyTest.shard2.replica_n4, service url = null, agent id = null] for registry solr.core.splitByRouteKeyTest.shard2.replica_n4 / com.codahale.metrics.MetricRegistry@100c9e6
   [junit4]   2> 2436592 INFO  (coreCloseExecutor-4136-thread-4) [n:127.0.0.1:40525_ c:splitByRouteKeyTest s:shard2 r:core_node7 x:splitByRouteKeyTest_shard2_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.splitByRouteKeyTest.shard2.leader, tag=15057616
   [junit4]   2> 2436593 INFO  (TEST-ShardSplitTest.test-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.Overseer Overseer (id=99395425039155218-127.0.0.1:40525_-n_0000000004) closing
   [junit4]   2> 2436594 WARN  (OverseerAutoScalingTriggerThread-99395425039155218-127.0.0.1:40525_-n_0000000004) [n:127.0.0.1:40525_    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 2436594 INFO  (OverseerStateUpdate-99395425039155218-127.0.0.1:40525_-n_0000000004) [n:127.0.0.1:40525_    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40525_
   [junit4]   2> 2437518 WARN  (zkCallback-2039-thread-5-processing-n:127.0.0.1:40525_) [n:127.0.0.1:40525_ c:splitByRouteKeyTest s:shard1_2 r:core_node25 x:splitByRouteKeyTest_shard1_2_replica0] o.a.s.c.SyncStrategy Closed, skipping sync up.
   [junit4]   2> 2437519 INFO  (zkCallback-2039-thread-5-processing-n:127.0.0.1:40525_) [n:127.0.0.1:40525_ c:splitByRouteKeyTest s:shard1_2 r:core_node25 x:splitByRouteKeyTest_shard1_2_replica0] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 2437519 INFO  (zkCallback-2039-thread-5-processing-n:127.0.0.1:40525_) [n:127.0.0.1:40525_ c:splitByRouteKeyTest s:shard1_2 r:core_node25 x:splitByRouteKeyTest_shard1_2_replica0] o.a.s.c.SolrCore [splitByRouteKeyTest_shard1_2_replica0]  CLOSING SolrCore org.apache.solr.core.SolrCore@ac4705
   [junit4]   2> 2437519 INFO  (zkCallback-2039-thread-5-processing-n:127.0.0.1:40525_) [n:127.0.0.1:40525_ c:splitByRouteKeyTest s:shard1_2 r:core_node25 x:splitByRouteKeyTest_shard1_2_replica0] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.splitByRouteKeyTest.shard1_2.replica0, tag=11290373
   [junit4]   2> 2437519 INFO  (zkCallback-2039-thread-5-processing-n:127.0.0.1:40525_) [n:127.0.0.1:40525_ c:splitByRouteKeyTest s:shard1_2 r:core_node25 x:splitByRouteKeyTest_shard1_2_replica0] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1c4c3d6: rootName = null, domain = solr.core.splitByRouteKeyTest.shard1_2.replica0, service url = null, agent id = null] for registry solr.core.splitByRouteKeyTest.shard1_2.replica0 / com.codahale.metrics.MetricRegistry@48292a
   [junit4]   2> 2437543 INFO  (zkCallback-2039-thread-5-processing-n:127.0.0.1:40525_) [n:127.0.0.1:40525_ c:splitByRouteKeyTest s:shard1_2 r:core_node25 x:splitByRouteKeyTest_shard1_2_replica0] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.splitByRouteKeyTest.shard1_2.leader, tag=11290373
   [junit4]   2> 2437547 INFO  (TEST-ShardSplitTest.test-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@17a5538{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 2437547 INFO  (TEST-ShardSplitTest.test-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@749e92{/,null,UNAVAILABLE}
   [junit4]   2> 2437547 INFO  (TEST-ShardSplitTest.test-seed#[E0877BC0E61808C1]) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 2437549 INFO  (TEST-ShardSplitTest.test-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.ChaosMonkey monkey: stop jetty! 45887
   [junit4]   2> 2437549 ERROR (TEST-ShardSplitTest.test-seed#[E0877BC0E61808C1]) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2437550 INFO  (TEST-ShardSplitTest.test-seed#[E0877BC0E61808C1]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36699 36699
   [junit4]   2> 2447758 INFO  (Thread-5172) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36699 36699
   [junit4]   2> 2447760 WARN  (Thread-5172) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 10 /solr/aliases.json
   [junit4]   2> 5 /solr/security.json
   [junit4]   2> 5 /solr/configs/conf1
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 159 /solr/collections/splitByRouteKeyTest/state.json
   [junit4]   2> 77 /solr/collections/routeFieldColl/state.json
   [junit4]   2> 10 /solr/clusterprops.json
   [junit4]   2> 10 /solr/clusterstate.json
   [junit4]   2> 5 /solr/autoscaling.json
   [junit4]   2> 4 /solr/collections/collection1/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 10 /solr/live_nodes
   [junit4]   2> 10 /solr/collections
   [junit4]   2> 5 /solr/overseer/queue
   [junit4]   2> 5 /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 5 /solr/overseer/collection-queue-work
   [junit4]   2> 5 /solr/overseer/queue-work
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.ShardSplitTest_E0877BC0E61808C1-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {rnd_b=PostingsFormat(name=Memory), shard_s=BlockTreeOrds(blocksize=128), a_t=BlockTreeOrds(blocksize=128), id=PostingsFormat(name=Memory)}, docValues:{rnd_b=DocValuesFormat(name=Lucene70), shard_s=DocValuesFormat(name=Lucene70), _version_=DocValuesFormat(name=Asserting), n_ti=DocValuesFormat(name=Lucene70), a_t=DocValuesFormat(name=Lucene70), a_i=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=145, maxMBSortInHeap=7.164215619201832, sim=RandomSimilarity(queryNorm=false): {}, locale=nl-NL, timezone=SystemV/PST8
   [junit4]   2> NOTE: Linux 4.13.0-26-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=177424040,total=512364544
   [junit4]   2> NOTE: All tests run in this JVM: [PreAnalyzedFieldTest, TestPushWriter, ConvertedLegacyTest, TestValueSourceCache, SpatialFilterTest, FileBasedSpellCheckerTest, NumberUtilsTest, ConfigureRecoveryStrategyTest, TestSchemaResource, CircularListTest, TestPullReplica, TimeZoneUtilsTest, SparseHLLTest, TestRecoveryHdfs, TestInfoStreamLogging, TestSerializedLuceneMatchVersion, SolrMetricReporterTest, SearchRateTriggerTest, ClusterStateTest, TestUtils, InfoHandlerTest, TestQuerySenderListener, TestQueryTypes, UtilsToolTest, TestAnalyzedSuggestions, TestRTimerTree, CdcrReplicationDistributedZkTest, TestDFRSimilarityFactory, TestLeaderElectionZkExpiry, TestCloudNestedDocsSort, RecoveryAfterSoftCommitTest, TestExactStatsCache, CloneFieldUpdateProcessorFactoryTest, TestStressInPlaceUpdates, TestSchemalessBufferedUpdates, SearchHandlerTest, HdfsRecoverLeaseTest, TestStressReorder, ParsingFieldUpdateProcessorsTest, TestZkAclsWithHadoopAuth, TestDelegationWithHadoopAuth, HdfsChaosMonkeyNothingIsSafeTest, TriggerIntegrationTest, TestSimpleQParserPlugin, TestFiltering, TestScoreJoinQPScore, TestClusterStateMutator, TestMacros, TestDownShardTolerantSearch, TestTolerantSearch, TestSuggestSpellingConverter, TestTolerantUpdateProcessorCloud, SpellCheckComponentTest, DistributedExpandComponentTest, OverseerStatusTest, CdcrVersionReplicationTest, BasicDistributedZk2Test, RegexBoostProcessorTest, TestHighFrequencyDictionaryFactory, TestSolrConfigHandler, SolrGraphiteReporterTest, TestPKIAuthenticationPlugin, TestUpdate, DistributedFacetPivotLongTailTest, ReplicationFactorTest, TestCloudRecovery, OverseerCollectionConfigSetProcessorTest, MultiTermTest, TestInPlaceUpdatesStandalone, ImplicitSnitchTest, TestStressUserVersions, TestExpandComponent, DistribJoinFromCollectionTest, AutoAddReplicasPlanActionTest, MetricTriggerTest, TestSolrCloudWithKerberosAlt, URLClassifyProcessorTest, ExecutePlanActionTest, TestSizeLimitedDistributedMap, ConnectionReuseTest, TestDistributedMissingSort, DateFieldTest, TestFieldCacheWithThreads, TestApiFramework, CursorMarkTest, CopyFieldTest, TestGeoJSONResponseWriter, TestNumericTerms64, TestCloudJSONFacetJoinDomain, ExitableDirectoryReaderTest, LeaderElectionIntegrationTest, ReplicaListTransformerTest, DirectSolrSpellCheckerTest, SolrSlf4jReporterTest, TestUnifiedSolrHighlighter, TestFaceting, TestLegacyNumericRangeQueryBuilder, UniqFieldsUpdateProcessorFactoryTest, HdfsUnloadDistributedZkTest, SSLMigrationTest, TestMultiValuedNumericRangeQuery, ClassificationUpdateProcessorFactoryTest, LeaderInitiatedRecoveryOnCommitTest, TestFastWriter, TestCustomSort, TestCodecSupport, ReturnFieldsTest, ShardSplitTest]
   [junit4] Completed [466/767 (1!)] on J2 in 938.49s, 5 tests, 1 failure <<< FAILURES!

[...truncated 45003 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk1.8.0_144) - Build # 1224 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1224/
Java: 64bit/jdk1.8.0_144 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:     1) Thread[id=9005, name=jetty-launcher-1530-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)    2) Thread[id=9010, name=jetty-launcher-1530-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
   1) Thread[id=9005, name=jetty-launcher-1530-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
        at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
        at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
        at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
        at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
        at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   2) Thread[id=9010, name=jetty-launcher-1530-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
        at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
        at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
        at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
        at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
        at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
        at __randomizedtesting.SeedInfo.seed([4E2A7C12C4FEA102]:0)


FAILED:  org.apache.solr.cloud.TestAuthenticationFramework.testBasics

Error Message:
Error from server at https://127.0.0.1:43065/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Can not find: /solr/testcollection_shard1_replica_n3/update</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason: <pre>    Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.8.v20171121</a><hr/>  </body> </html>

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at https://127.0.0.1:43065/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n3/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.8.v20171121</a><hr/>

</body>
</html>

        at __randomizedtesting.SeedInfo.seed([4E2A7C12C4FEA102:73F2D23EFC10FF72]:0)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:550)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1013)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:946)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:946)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:946)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:946)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:946)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
        at org.apache.solr.cloud.TestAuthenticationFramework.collectionCreateSearchDeleteTwice(TestAuthenticationFramework.java:126)
        at org.apache.solr.cloud.TestAuthenticationFramework.testBasics(TestAuthenticationFramework.java:74)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:43065/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n3/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.8.v20171121</a><hr/>

</body>
</html>

        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:527)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        ... 1 more




Build Log:
[...truncated 11762 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestAuthenticationFramework
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_4E2A7C12C4FEA102-001/init-core-data-001
   [junit4]   2> 102961 WARN  (SUITE-TestAuthenticationFramework-seed#[4E2A7C12C4FEA102]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 102961 INFO  (SUITE-TestAuthenticationFramework-seed#[4E2A7C12C4FEA102]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 102962 INFO  (SUITE-TestAuthenticationFramework-seed#[4E2A7C12C4FEA102]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 102995 INFO  (SUITE-TestAuthenticationFramework-seed#[4E2A7C12C4FEA102]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 103000 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[4E2A7C12C4FEA102]) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_4E2A7C12C4FEA102-001/tempDir-001
   [junit4]   2> 103000 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[4E2A7C12C4FEA102]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 103000 INFO  (Thread-1760) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 103000 INFO  (Thread-1760) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 103001 ERROR (Thread-1760) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 103100 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[4E2A7C12C4FEA102]) [    ] o.a.s.c.ZkTestServer start zk server on port:36723
   [junit4]   2> 103103 INFO  (zkConnectionManagerCallback-112-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 103118 INFO  (jetty-launcher-109-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T10:27:37+13:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 103118 INFO  (jetty-launcher-109-thread-2) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T10:27:37+13:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 103119 INFO  (jetty-launcher-109-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 103119 INFO  (jetty-launcher-109-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 103119 INFO  (jetty-launcher-109-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 103119 INFO  (jetty-launcher-109-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 103119 INFO  (jetty-launcher-109-thread-2) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 103119 INFO  (jetty-launcher-109-thread-1) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 103122 INFO  (jetty-launcher-109-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@75b404ee{/solr,null,AVAILABLE}
   [junit4]   2> 103122 INFO  (jetty-launcher-109-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@79a3c45{/solr,null,AVAILABLE}
   [junit4]   2> 103137 INFO  (jetty-launcher-109-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5c0689af{SSL,[ssl, http/1.1]}{127.0.0.1:43065}
   [junit4]   2> 103137 INFO  (jetty-launcher-109-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5743a340{SSL,[ssl, http/1.1]}{127.0.0.1:33451}
   [junit4]   2> 103137 INFO  (jetty-launcher-109-thread-1) [    ] o.e.j.s.Server Started @104718ms
   [junit4]   2> 103137 INFO  (jetty-launcher-109-thread-2) [    ] o.e.j.s.Server Started @104718ms
   [junit4]   2> 103137 INFO  (jetty-launcher-109-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43065}
   [junit4]   2> 103137 ERROR (jetty-launcher-109-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 103138 INFO  (jetty-launcher-109-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.3.0
   [junit4]   2> 103138 INFO  (jetty-launcher-109-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 103138 INFO  (jetty-launcher-109-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 103138 INFO  (jetty-launcher-109-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-22T23:13:02.863Z
   [junit4]   2> 103137 INFO  (jetty-launcher-109-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33451}
   [junit4]   2> 103139 ERROR (jetty-launcher-109-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 103139 INFO  (jetty-launcher-109-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.3.0
   [junit4]   2> 103139 INFO  (jetty-launcher-109-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 103139 INFO  (jetty-launcher-109-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 103139 INFO  (jetty-launcher-109-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-22T23:13:02.864Z
   [junit4]   2> 103140 INFO  (zkConnectionManagerCallback-114-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 103140 INFO  (jetty-launcher-109-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 103141 INFO  (zkConnectionManagerCallback-116-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 103141 INFO  (jetty-launcher-109-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 103152 INFO  (jetty-launcher-109-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36723/solr
   [junit4]   2> 103153 INFO  (jetty-launcher-109-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36723/solr
   [junit4]   2> 103155 INFO  (zkConnectionManagerCallback-121-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 103155 INFO  (zkConnectionManagerCallback-124-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 103163 INFO  (zkConnectionManagerCallback-126-thread-1-processing-n:127.0.0.1:33451_solr) [n:127.0.0.1:33451_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 103164 INFO  (zkConnectionManagerCallback-128-thread-1-processing-n:127.0.0.1:43065_solr) [n:127.0.0.1:43065_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 103208 INFO  (jetty-launcher-109-thread-2) [n:127.0.0.1:33451_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 103208 INFO  (jetty-launcher-109-thread-1) [n:127.0.0.1:43065_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 103208 INFO  (jetty-launcher-109-thread-2) [n:127.0.0.1:33451_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33451_solr
   [junit4]   2> 103208 INFO  (jetty-launcher-109-thread-1) [n:127.0.0.1:43065_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43065_solr
   [junit4]   2> 103208 INFO  (jetty-launcher-109-thread-2) [n:127.0.0.1:33451_solr    ] o.a.s.c.Overseer Overseer (id=99396012128731141-127.0.0.1:33451_solr-n_0000000000) starting
   [junit4]   2> 103210 INFO  (zkCallback-127-thread-1-processing-n:127.0.0.1:43065_solr) [n:127.0.0.1:43065_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 103210 INFO  (zkCallback-125-thread-1-processing-n:127.0.0.1:33451_solr) [n:127.0.0.1:33451_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 103229 INFO  (jetty-launcher-109-thread-2) [n:127.0.0.1:33451_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33451_solr
   [junit4]   2> 103230 INFO  (zkCallback-125-thread-1-processing-n:127.0.0.1:33451_solr) [n:127.0.0.1:33451_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 103230 INFO  (zkCallback-127-thread-1-processing-n:127.0.0.1:43065_solr) [n:127.0.0.1:43065_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 103240 INFO  (jetty-launcher-109-thread-1) [n:127.0.0.1:43065_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
   [junit4]   2> 103280 INFO  (jetty-launcher-109-thread-2) [n:127.0.0.1:33451_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
   [junit4]   2> 103297 INFO  (jetty-launcher-109-thread-1) [n:127.0.0.1:43065_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43065.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@59827d75
   [junit4]   2> 103304 INFO  (jetty-launcher-109-thread-1) [n:127.0.0.1:43065_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43065.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@59827d75
   [junit4]   2> 103304 INFO  (jetty-launcher-109-thread-1) [n:127.0.0.1:43065_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43065.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@59827d75
   [junit4]   2> 103305 INFO  (jetty-launcher-109-thread-1) [n:127.0.0.1:43065_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_4E2A7C12C4FEA102-001/tempDir-001/node1/.
   [junit4]   2> 103306 INFO  (jetty-launcher-109-thread-2) [n:127.0.0.1:33451_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33451.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@59827d75
   [junit4]   2> 103312 INFO  (jetty-launcher-109-thread-2) [n:127.0.0.1:33451_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33451.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@59827d75
   [junit4]   2> 103313 INFO  (jetty-launcher-109-thread-2) [n:127.0.0.1:33451_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33451.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@59827d75
   [junit4]   2> 103313 INFO  (jetty-launcher-109-thread-2) [n:127.0.0.1:33451_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_4E2A7C12C4FEA102-001/tempDir-001/node2/.
   [junit4]   2> 103357 INFO  (zkConnectionManagerCallback-134-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 103364 INFO  (zkConnectionManagerCallback-138-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 103365 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[4E2A7C12C4FEA102]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 103365 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[4E2A7C12C4FEA102]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36723/solr ready
   [junit4]   2> 103366 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[4E2A7C12C4FEA102]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testBasics
   [junit4]   2> 103449 INFO  (qtp854853172-2252) [n:127.0.0.1:43065_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=2&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 103451 INFO  (OverseerThreadFactory-330-thread-1-processing-n:127.0.0.1:33451_solr) [n:127.0.0.1:33451_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection testcollection
   [junit4]   2> 103563 INFO  (OverseerStateUpdate-99396012128731141-127.0.0.1:33451_solr-n_0000000000) [n:127.0.0.1:33451_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:33451/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 103566 INFO  (OverseerStateUpdate-99396012128731141-127.0.0.1:33451_solr-n_0000000000) [n:127.0.0.1:33451_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n3",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:43065/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 103572 INFO  (OverseerStateUpdate-99396012128731141-127.0.0.1:33451_solr-n_0000000000) [n:127.0.0.1:33451_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n5",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:33451/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 103575 INFO  (OverseerStateUpdate-99396012128731141-127.0.0.1:33451_solr-n_0000000000) [n:127.0.0.1:33451_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n7",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:43065/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 103873 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard1_replica_n3&action=CREATE&numShards=2&collection=testcollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 103873 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 103884 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard2_replica_n5&action=CREATE&numShards=2&collection=testcollection&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 103885 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 103895 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard2_replica_n7&action=CREATE&numShards=2&collection=testcollection&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 103909 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&collection=testcollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 104012 INFO  (zkCallback-127-thread-1-processing-n:127.0.0.1:43065_solr) [n:127.0.0.1:43065_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 104012 INFO  (zkCallback-125-thread-1-processing-n:127.0.0.1:33451_solr) [n:127.0.0.1:33451_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 104014 INFO  (zkCallback-125-thread-2-processing-n:127.0.0.1:33451_solr) [n:127.0.0.1:33451_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 104908 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 104915 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.s.IndexSchema [testcollection_shard2_replica_n7] Schema name=minimal
   [junit4]   2> 104917 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 104918 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard2_replica_n7' using configuration from collection testcollection, trusted=true
   [junit4]   2> 104918 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43065.solr.core.testcollection.shard2.replica_n7' (registry 'solr.core.testcollection.shard2.replica_n7') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@59827d75
   [junit4]   2> 104918 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 104918 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.c.SolrCore [[testcollection_shard2_replica_n7] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_4E2A7C12C4FEA102-001/tempDir-001/node1/testcollection_shard2_replica_n7], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_4E2A7C12C4FEA102-001/tempDir-001/node1/./testcollection_shard2_replica_n7/data/]
   [junit4]   2> 104920 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 104973 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 104977 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.s.IndexSchema [testcollection_shard1_replica_n3] Schema name=minimal
   [junit4]   2> 104979 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 104979 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n3' using configuration from collection testcollection, trusted=true
   [junit4]   2> 104979 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.s.IndexSchema [testcollection_shard2_replica_n5] Schema name=minimal
   [junit4]   2> 104979 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43065.solr.core.testcollection.shard1.replica_n3' (registry 'solr.core.testcollection.shard1.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@59827d75
   [junit4]   2> 104979 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 104980 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.SolrCore [[testcollection_shard1_replica_n3] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_4E2A7C12C4FEA102-001/tempDir-001/node1/testcollection_shard1_replica_n3], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_4E2A7C12C4FEA102-001/tempDir-001/node1/./testcollection_shard1_replica_n3/data/]
   [junit4]   2> 104981 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 104981 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard2_replica_n5' using configuration from collection testcollection, trusted=true
   [junit4]   2> 104982 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33451.solr.core.testcollection.shard2.replica_n5' (registry 'solr.core.testcollection.shard2.replica_n5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@59827d75
   [junit4]   2> 104982 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 104982 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.SolrCore [[testcollection_shard2_replica_n5] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_4E2A7C12C4FEA102-001/tempDir-001/node2/testcollection_shard2_replica_n5], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_4E2A7C12C4FEA102-001/tempDir-001/node2/./testcollection_shard2_replica_n5/data/]
   [junit4]   2> 104984 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 105002 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema [testcollection_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 105004 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 105004 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n1' using configuration from collection testcollection, trusted=true
   [junit4]   2> 105004 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33451.solr.core.testcollection.shard1.replica_n1' (registry 'solr.core.testcollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@59827d75
   [junit4]   2> 105004 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 105004 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [[testcollection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_4E2A7C12C4FEA102-001/tempDir-001/node2/testcollection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_4E2A7C12C4FEA102-001/tempDir-001/node2/./testcollection_shard1_replica_n1/data/]
   [junit4]   2> 105053 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 105053 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 105053 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 105053 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 105054 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 105054 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 105055 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 105055 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 105056 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@642314ae[testcollection_shard1_replica_n3] main]
   [junit4]   2> 105056 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.s.SolrIndexSearcher Opening [Searcher@6306c480[testcollection_shard2_replica_n5] main]
   [junit4]   2> 105056 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 105057 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 105057 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 105059 INFO  (searcherExecutor-336-thread-1-processing-n:127.0.0.1:43065_solr x:testcollection_shard1_replica_n3 s:shard1 c:testcollection r:core_node4) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.SolrCore [testcollection_shard1_replica_n3] Registered new searcher Searcher@642314ae[testcollection_shard1_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 105059 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 105059 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590336196217667584
   [junit4]   2> 105060 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 105060 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 105062 INFO  (searcherExecutor-337-thread-1-processing-n:127.0.0.1:33451_solr x:testcollection_shard2_replica_n5 s:shard2 c:testcollection r:core_node6) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.SolrCore [testcollection_shard2_replica_n5] Registered new searcher Searcher@6306c480[testcollection_shard2_replica_n5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 105062 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590336196220813312
   [junit4]   2> 105066 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 105068 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 105094 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 105094 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 105094 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 105094 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 105095 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 105095 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 105095 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 105095 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 105096 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.s.SolrIndexSearcher Opening [Searcher@4727613d[testcollection_shard2_replica_n7] main]
   [junit4]   2> 105096 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7295a0[testcollection_shard1_replica_n1] main]
   [junit4]   2> 105097 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 105097 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 105097 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 105097 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 105097 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 105097 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 105097 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590336196257513472
   [junit4]   2> 105097 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590336196257513472
   [junit4]   2> 105099 INFO  (searcherExecutor-338-thread-1-processing-n:127.0.0.1:33451_solr x:testcollection_shard1_replica_n1 s:shard1 c:testcollection r:core_node2) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [testcollection_shard1_replica_n1] Registered new searcher Searcher@7295a0[testcollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 105099 INFO  (searcherExecutor-335-thread-1-processing-n:127.0.0.1:43065_solr x:testcollection_shard2_replica_n7 s:shard2 c:testcollection r:core_node8) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.c.SolrCore [testcollection_shard2_replica_n7] Registered new searcher Searcher@4727613d[testcollection_shard2_replica_n7] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 105169 INFO  (zkCallback-125-thread-1-processing-n:127.0.0.1:33451_solr) [n:127.0.0.1:33451_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 105169 INFO  (zkCallback-127-thread-1-processing-n:127.0.0.1:43065_solr) [n:127.0.0.1:43065_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 105169 INFO  (zkCallback-125-thread-2-processing-n:127.0.0.1:33451_solr) [n:127.0.0.1:33451_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 105567 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 105567 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 105567 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:43065/solr/testcollection_shard1_replica_n3/
   [junit4]   2> 105567 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=testcollection_shard1_replica_n3 url=https://127.0.0.1:43065/solr START replicas=[https://127.0.0.1:33451/solr/testcollection_shard1_replica_n1/] nUpdates=100
   [junit4]   2> 105568 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 105568 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 105569 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:33451/solr/testcollection_shard2_replica_n5/
   [junit4]   2> 105570 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.PeerSync PeerSync: core=testcollection_shard2_replica_n5 url=https://127.0.0.1:33451/solr START replicas=[https://127.0.0.1:43065/solr/testcollection_shard2_replica_n7/] nUpdates=100
   [junit4]   2> 105580 INFO  (qtp854853172-2311) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.c.S.Request [testcollection_shard2_replica_n7]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 105580 INFO  (qtp561894499-2241) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.c.S.Request [testcollection_shard1_replica_n1]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 105868 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=testcollection_shard1_replica_n3 url=https://127.0.0.1:43065/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 105868 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 105868 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 105868 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 105870 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:43065/solr/testcollection_shard1_replica_n3/ shard1
   [junit4]   2> 105871 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.PeerSync PeerSync: core=testcollection_shard2_replica_n5 url=https://127.0.0.1:33451/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 105871 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 105871 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 105871 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 105872 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:33451/solr/testcollection_shard2_replica_n5/ shard2
   [junit4]   2> 105974 INFO  (zkCallback-127-thread-1-processing-n:127.0.0.1:43065_solr) [n:127.0.0.1:43065_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 105974 INFO  (zkCallback-125-thread-2-processing-n:127.0.0.1:33451_solr) [n:127.0.0.1:33451_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 105974 INFO  (zkCallback-125-thread-1-processing-n:127.0.0.1:33451_solr) [n:127.0.0.1:33451_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 106021 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 106022 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard1_replica_n3&action=CREATE&numShards=2&collection=testcollection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2150
   [junit4]   2> 106023 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 106024 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard2_replica_n5&action=CREATE&numShards=2&collection=testcollection&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2141
   [junit4]   2> 106102 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&collection=testcollection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2194
   [junit4]   2> 106102 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard2_replica_n7&action=CREATE&numShards=2&collection=testcollection&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2215
   [junit4]   2> 106106 INFO  (qtp854853172-2252) [n:127.0.0.1:43065_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 106204 INFO  (zkCallback-127-thread-1-processing-n:127.0.0.1:43065_solr) [n:127.0.0.1:43065_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 106204 INFO  (zkCallback-125-thread-1-processing-n:127.0.0.1:33451_solr) [n:127.0.0.1:33451_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 106204 INFO  (zkCallback-125-thread-2-processing-n:127.0.0.1:33451_solr) [n:127.0.0.1:33451_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 107106 INFO  (qtp854853172-2252) [n:127.0.0.1:43065_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=2&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=3657
   [junit4]   2> 107107 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[4E2A7C12C4FEA102]) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: testcollection failOnTimeout:true timeout (sec):330
   [junit4]   1> -
   [junit4]   1> replica:core_node2 rstate:active live:true
   [junit4]   1> replica:core_node4 rstate:active live:true
   [junit4]   1> replica:core_node6 rstate:active live:true
   [junit4]   1> replica:core_node8 rstate:active live:true
   [junit4]   1> no one is recoverying
   [junit4]   2> 107108 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[4E2A7C12C4FEA102]) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: testcollection
   [junit4]   2> 107142 INFO  (qtp854853172-2250) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.c.S.Request [testcollection_shard2_replica_n7]  webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testcollection:5&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://127.0.0.1:43065/solr/testcollection_shard2_replica_n7/|https://127.0.0.1:33451/solr/testcollection_shard2_replica_n5/&rows=10&version=2&q=*:*&NOW=1516662786848&isShard=true&wt=javabin} hits=0 status=0 QTime=1
   [junit4]   2> 107142 INFO  (qtp854853172-2240) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.S.Request [testcollection_shard1_replica_n3]  webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testcollection:5&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://127.0.0.1:43065/solr/testcollection_shard1_replica_n3/|https://127.0.0.1:33451/solr/testcollection_shard1_replica_n1/&rows=10&version=2&q=*:*&NOW=1516662786848&isShard=true&wt=javabin} hits=0 status=0 QTime=0
   [junit4]   2> 107150 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.S.Request [testcollection_shard2_replica_n5]  webapp=/solr path=/select params={q=*:*&_stateVer_=testcollection:5&wt=javabin&version=2} hits=0 status=0 QTime=28
   [junit4]   2> 107215 INFO  (qtp561894499-2307) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:43065/solr/testcollection_shard1_replica_n3/&wt=javabin&version=2}{add=[1 (1590336198443794432)]} 0 6
   [junit4]   2> 107219 INFO  (qtp854853172-2248) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n3]  webapp=/solr path=/update params={_stateVer_=testcollection:5&wt=javabin&version=2}{add=[1 (1590336198443794432)]} 0 51
   [junit4]   2> 107243 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1590336198507757568,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 107243 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@251d434e commitCommandVersion:1590336198507757568
   [junit4]   2> 107273 INFO  (qtp854853172-2250) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1590336198539214848,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 107277 INFO  (qtp854853172-2250) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 107278 INFO  (qtp854853172-2250) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 107278 INFO  (qtp854853172-2250) [n:127.0.0.1:43065_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard2_replica_n7]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:43065/solr/testcollection_shard1_replica_n3/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 5
   [junit4]   2> 107287 INFO  (qtp561894499-2249) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1590336198553894912,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 107297 INFO  (qtp854853172-2252) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1590336198564380672,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 107297 INFO  (qtp854853172-2252) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@34af4f2e commitCommandVersion:1590336198564380672
   [junit4]   2> 107297 INFO  (qtp561894499-2249) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 107300 INFO  (qtp561894499-2249) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 107300 INFO  (qtp561894499-2249) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard2_replica_n5]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:43065/solr/testcollection_shard1_replica_n3/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 13
   [junit4]   2> 107346 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7d68b929[testcollection_shard1_replica_n1] main]
   [junit4]   2> 107347 INFO  (qtp854853172-2252) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@3e733aa[testcollection_shard1_replica_n3] main]
   [junit4]   2> 107348 INFO  (searcherExecutor-338-thread-1-processing-n:127.0.0.1:33451_solr x:testcollection_shard1_replica_n1 s:shard1 c:testcollection r:core_node2) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [testcollection_shard1_replica_n1] Registered new searcher Searcher@7d68b929[testcollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.3.0):C1)))}
   [junit4]   2> 107349 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 107349 INFO  (qtp561894499-2251) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:43065/solr/testcollection_shard1_replica_n3/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 106
   [junit4]   2> 107355 INFO  (searcherExecutor-336-thread-1-processing-n:127.0.0.1:43065_solr x:testcollection_shard1_replica_n3 s:shard1 c:testcollection r:core_node4) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.SolrCore [testcollection_shard1_replica_n3] Registered new searcher Searcher@3e733aa[testcollection_shard1_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.3.0):C1)))}
   [junit4]   2> 107359 INFO  (qtp854853172-2252) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 107359 INFO  (qtp854853172-2252) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n3]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:43065/solr/testcollection_shard1_replica_n3/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 62
   [junit4]   2> 107360 INFO  (qtp854853172-2248) [n:127.0.0.1:43065_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n3]  webapp=/solr path=/update params={_stateVer_=testcollection:5&commit=true&wt=javabin&version=2}{commit=} 0 127
   [junit4]   2> 107376 INFO  (qtp561894499-2241) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.S.Request [testcollection_shard2_replica_n5]  webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testcollection:5&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://127.0.0.1:33451/solr/testcollection_shard2_replica_n5/|https://127.0.0.1:43065/solr/testcollection_shard2_replica_n7/&rows=10&version=2&q=*:*&NOW=1516662787088&isShard=true&wt=javabin} hits=0 status=0 QTime=0
   [junit4]   2> 107379 INFO  (qtp561894499-2305) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.c.S.Request [testcollection_shard1_replica_n1]  webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testcollection:5&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://127.0.0.1:33451/solr/testcollection_shard1_replica_n1/|https://127.0.0.1:43065/solr/testcollection_shard1_replica_n3/&rows=10&version=2&q=*:*&NOW=1516662787088&isShard=true&wt=javabin} hits=1 status=0 QTime=1
   [junit4]   2> 107385 INFO  (qtp561894499-2249) [n:127.0.0.1:33451_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.c.S.Request [testcollection_shard1_replica_n1]  webapp=/solr path=/select params={q=*:*&df=text&distrib=false&_stateVer_=testcollection:5&shards.purpose=64&NOW=1516662787088&ids=1&isShard=true&shard.url=https://127.0.0.1:33451/solr/testcollection_shard1_replica_n1/|https://127.0.0.1:43065/solr/testcollection_shard1_replica_n3/&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 107397 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.S.Request [testcollection_shard2_replica_n5]  webapp=/solr path=/select params={q=*:*&_stateVer_=testcollection:5&wt=javabin&version=2} hits=1 status=0 QTime=33
   [junit4]   2> 107400 INFO  (qtp854853172-2250) [n:127.0.0.1:43065_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=testcollection&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 107427 INFO  (OverseerCollectionConfigSetProcessor-99396012128731141-127.0.0.1:33451_solr-n_0000000000) [n:127.0.0.1:33451_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 107430 INFO  (OverseerThreadFactory-330-thread-2-processing-n:127.0.0.1:33451_solr) [n:127.0.0.1:33451_solr    ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 107431 INFO  (qtp561894499-2305) [n:127.0.0.1:33451_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard1.replica_n1, tag=null
   [junit4]   2> 107431 INFO  (qtp561894499-2305) [n:127.0.0.1:33451_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3edda047: rootName = solr_33451, domain = solr.core.testcollection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.testcollection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@1be00838
   [junit4]   2> 107447 INFO  (qtp854853172-2240) [n:127.0.0.1:43065_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard2.replica_n7, tag=null
   [junit4]   2> 107448 INFO  (qtp854853172-2240) [n:127.0.0.1:43065_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4f49bd48: rootName = solr_43065, domain = solr.core.testcollection.shard2.replica_n7, service url = null, agent id = null] for registry solr.core.testcollection.shard2.replica_n7 / com.codahale.metrics.MetricRegistry@1a2098cb
   [junit4]   2> 107463 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard1.replica_n3, tag=null
   [junit4]   2> 107463 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6897251: rootName = solr_43065, domain = solr.core.testcollection.shard1.replica_n3, service url = null, agent id = null] for registry solr.core.testcollection.shard1.replica_n3 / com.codahale.metrics.MetricRegistry@fe015aa
   [junit4]   2> 107475 INFO  (qtp854853172-2240) [n:127.0.0.1:43065_solr    ] o.a.s.c.SolrCore [testcollection_shard2_replica_n7]  CLOSING SolrCore org.apache.solr.core.SolrCore@5222ab71
   [junit4]   2> 107477 INFO  (qtp854853172-2240) [n:127.0.0.1:43065_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard2.replica_n7, tag=1378003825
   [junit4]   2> 107477 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr    ] o.a.s.c.SolrCore [testcollection_shard1_replica_n3]  CLOSING SolrCore org.apache.solr.core.SolrCore@53d07cf5
   [junit4]   2> 107477 INFO  (qtp854853172-2240) [n:127.0.0.1:43065_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testcollection.shard2.leader, tag=1378003825
   [junit4]   2> 107487 INFO  (qtp561894499-2241) [n:127.0.0.1:33451_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard2.replica_n5, tag=null
   [junit4]   2> 107487 INFO  (qtp561894499-2305) [n:127.0.0.1:33451_solr    ] o.a.s.c.SolrCore [testcollection_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@5a0ded40
   [junit4]   2> 107487 INFO  (qtp561894499-2241) [n:127.0.0.1:33451_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2ebed9cf: rootName = solr_33451, domain = solr.core.testcollection.shard2.replica_n5, service url = null, agent id = null] for registry solr.core.testcollection.shard2.replica_n5 / com.codahale.metrics.MetricRegistry@f1256f8
   [junit4]   2> 107507 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard1.replica_n3, tag=1406172405
   [junit4]   2> 107507 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testcollection.shard1.leader, tag=1406172405
   [junit4]   2> 107510 INFO  (qtp561894499-2305) [n:127.0.0.1:33451_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard1.replica_n1, tag=1510862144
   [junit4]   2> 107510 INFO  (qtp561894499-2241) [n:127.0.0.1:33451_solr    ] o.a.s.c.SolrCore [testcollection_shard2_replica_n5]  CLOSING SolrCore org.apache.solr.core.SolrCore@5426d48b
   [junit4]   2> 107510 INFO  (qtp561894499-2305) [n:127.0.0.1:33451_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testcollection.shard1.leader, tag=1510862144
   [junit4]   2> 107510 INFO  (qtp561894499-2241) [n:127.0.0.1:33451_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard2.replica_n5, tag=1411830923
   [junit4]   2> 107510 INFO  (qtp561894499-2241) [n:127.0.0.1:33451_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testcollection.shard2.leader, tag=1411830923
   [junit4]   2> 107534 INFO  (qtp854853172-2240) [n:127.0.0.1:43065_solr    ] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration.
   [junit4]   2> 107541 INFO  (qtp854853172-2240) [n:127.0.0.1:43065_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=testcollection_shard2_replica_n7&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=94
   [junit4]   2> 107559 WARN  (zkCallback-127-thread-1-processing-n:127.0.0.1:43065_solr) [n:127.0.0.1:43065_solr    ] o.a.s.c.LeaderElector Our node is no longer in line to be leader
   [junit4]   2> 107560 INFO  (qtp561894499-2241) [n:127.0.0.1:33451_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=testcollection_shard2_replica_n5&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=123
   [junit4]   2> 107590 INFO  (qtp561894499-2305) [n:127.0.0.1:33451_solr    ] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration.
   [junit4]   2> 107591 WARN  (zkCallback-125-thread-1-processing-n:127.0.0.1:33451_solr) [n:127.0.0.1:33451_solr    ] o.a.s.c.LeaderElector Our node is no longer in line to be leader
   [junit4]   2> 107592 INFO  (qtp561894499-2305) [n:127.0.0.1:33451_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=testcollection_shard1_replica_n1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=161
   [junit4]   2> 107592 INFO  (qtp854853172-2308) [n:127.0.0.1:43065_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=testcollection_shard1_replica_n3&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=145
   [junit4]   2> 108300 INFO  (qtp854853172-2250) [n:127.0.0.1:43065_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=testcollection&action=DELETE&wt=javabin&version=2} status=0 QTime=900
   [junit4]   2> 108301 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[4E2A7C12C4FEA102]) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for collection to disappear - collection: testcollection failOnTimeout:true timeout (sec):330
   [junit4]   1> -
   [junit4]   2> 108301 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[4E2A7C12C4FEA102]) [    ] o.a.s.c.AbstractDistribZkTestBase Collection has disappeared - collection: testcollection
   [junit4]   2> 108302 INFO  (qtp854853172-2252) [n:127.0.0.1:43065_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=2&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 108303 INFO  (OverseerThreadFactory-330-thread-3-processing-n:127.0.0.1:33451_solr) [n:127.0.0.1:33451_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection testcollection
   [junit4]   2> 108303 INFO  (OverseerCollectionConfigSetProcessor-99396012128731141-127.0.0.1:33451_solr-n_0000000000) [n:127.0.0.1:33451_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 108507 INFO  (OverseerStateUpdate-99396012128731141-127.0.0.1:33451_solr-n_0000000000) [n:127.0.0.1:33451_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:43065/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 108508 INFO  (OverseerStateUpdate-99396012128731141-127.0.0.1:33451_solr-n_0000000000) [n:127.0.0.1:33451_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:33451/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 108509 INFO  (OverseerStateUpdate-99396012128731141-127.0.0.1:33451_solr-n_0000000000) [n:127.0.0.1:33451_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:43065/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 108510 INFO  (OverseerStateUpdate-99396012128731141-127.0.0.1:33451_solr-n_0000000000) [n:127.0.0.1:33451_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n6",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:33451/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 108713 INFO  (qtp854853172-2242) [n:127.0.0.1:43065_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&collection=testcollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 108713 INFO  (qtp854853172-2248) [n:127.0.0.1:43065_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard2_replica_n4&action=CREATE&numShards=2&collection=testcollection&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 108714 INFO  (qtp561894499-2310) [n:127.0.0.1:33451_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard2_replica_n6&action=CREATE&numShards=2&collection=testcollection&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 108714 INFO  (qtp561894499-2249) [n:127.0.0.1:33451_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=solrCloudCollectionConfig&newCollection=true&name=testcollection_shard1_replica_n2&action=CREATE&numShards=2&collection=testcollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 108839 INFO  (zkCallback-127-thread-1-processing-n:127.0.0.1:43065_solr) [n:127.0.0.1:43065_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 108839 INFO  (zkCallback-127-thread-2-processing-n:127.0.0.1:43065_solr) [n:127.0.0.1:43065_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 108840 INFO  (zkCallback-125-thread-2-processing-n:127.0.0.1:33451_solr) [n:127.0.0.1:33451_solr  

[...truncated too long message...]

hronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
   [junit4]   2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
   [junit4]   2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
   [junit4]   2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
   [junit4]   2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
   [junit4]   2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
   [junit4]   2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
   [junit4]   2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> 1018280 ERROR (jetty-launcher-1530-thread-2-EventThread) [n:127.0.0.1:35623_solr    ] o.a.c.f.i.CuratorFrameworkImpl Watcher exception
   [junit4]   2> java.lang.InterruptedException
   [junit4]   2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
   [junit4]   2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
   [junit4]   2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
   [junit4]   2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
   [junit4]   2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
   [junit4]   2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
   [junit4]   2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
   [junit4]   2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
   [junit4]   2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> Jan 22, 2018 11:28:17 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=7242, maxDocsPerChunk=644, blockSize=311), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=7242, blockSize=311)), sim=RandomSimilarity(queryNorm=true): {}, locale=en-ZA, timezone=EET
   [junit4]   2> NOTE: Linux 4.13.0-26-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=8,threads=1,free=121175088,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [ChaosMonkeySafeLeaderWithPullReplicasTest, TestReplicationHandlerBackup, TestNamedUpdateProcessors, SuggestComponentTest, TestPseudoReturnFields, TestPullReplicaErrorHandling, TestJsonRequest, RequestHandlersTest, AutoScalingHandlerTest, TestHighlightDedupGrouping, BinaryUpdateRequestHandlerTest, CoreSorterTest, CursorPagingTest, HdfsThreadLeakTest, DeleteInactiveReplicaTest, TestRandomFaceting, TestSegmentSorting, TestSolrDeletionPolicy1, UUIDUpdateProcessorFallbackTest, HttpSolrCallGetCoreTest, TestHdfsCloudBackupRestore, TestQuerySenderNoQuery, RAMDirectoryFactoryTest, MissingSegmentRecoveryTest, TestImpersonationWithHadoopAuth, SolrRequestParserTest, TestConfigReload, HdfsWriteToMultipleCollectionsTest, DeleteReplicaTest, TestEmbeddedSolrServerSchemaAPI, OverseerModifyCollectionTest, TestHashQParserPlugin, NodeAddedTriggerTest, MBeansHandlerTest, TestReloadDeadlock, SpellCheckCollatorTest, TestSchemaSimilarityResource, FullSolrCloudDistribCmdsTest, SaslZkACLProviderTest, TestPerFieldSimilarityWithDefaultOverride, TestFieldCache, PKIAuthenticationIntegrationTest, TestDeleteCollectionOnDownNodes, TestPayloadScoreQParserPlugin, ForceLeaderTest, HighlighterConfigTest, LeaderFailoverAfterPartitionTest, TestCSVLoader, TestJoin, PreAnalyzedFieldManagedSchemaCloudTest, TestSolrFieldCacheBean, TestGraphMLResponseWriter, SolrMetricsIntegrationTest, TestUseDocValuesAsStored, XmlUpdateRequestHandlerTest, TestShardHandlerFactory, FieldAnalysisRequestHandlerTest, ZkCLITest, CheckHdfsIndexTest, BlockCacheTest, TestStressLucene, TestSolrCloudWithSecureImpersonation]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSolrCloudWithSecureImpersonation -Dtests.seed=4E2A7C12C4FEA102 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en-ZA -Dtests.timezone=EET -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | TestSolrCloudWithSecureImpersonation (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
   [junit4]    >    1) Thread[id=9005, name=jetty-launcher-1530-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
   [junit4]    >         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
   [junit4]    >         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
   [junit4]    >         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
   [junit4]    >         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]    >    2) Thread[id=9010, name=jetty-launcher-1530-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
   [junit4]    >         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
   [junit4]    >         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
   [junit4]    >         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
   [junit4]    >         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([4E2A7C12C4FEA102]:0)
   [junit4] Completed [238/767 (2!)] on J0 in 50.74s, 12 tests, 1 error <<< FAILURES!

[...truncated 49981 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk1.8.0_144) - Build # 1225 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1225/
Java: 64bit/jdk1.8.0_144 -XX:-UseCompressedOops -XX:+UseG1GC

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd

Error Message:
2 threads leaked from SUITE scope at org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd:     1) Thread[id=114, name=qtp245207537-114, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)    2) Thread[id=108, name=qtp245207537-108, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd:
   1) Thread[id=114, name=qtp245207537-114, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
   2) Thread[id=108, name=qtp245207537-108, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([EB24F3408972FCDB]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=114, name=qtp245207537-114, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)    2) Thread[id=108, name=qtp245207537-108, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=114, name=qtp245207537-114, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
   2) Thread[id=108, name=qtp245207537-108, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([EB24F3408972FCDB]:0)


FAILED:  org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory

Error Message:
expected:<5> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
        at __randomizedtesting.SeedInfo.seed([E1005774C8280AEF:8CFCF3897260F5E8]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:265)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13859 lines...]
   [junit4] Suite: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
   [junit4]   2> 3085828 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[E1005774C8280AEF]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/init-core-data-001
   [junit4]   2> 3085828 WARN  (SUITE-AutoscalingHistoryHandlerTest-seed#[E1005774C8280AEF]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=236 numCloses=236
   [junit4]   2> 3085829 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[E1005774C8280AEF]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 3085830 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[E1005774C8280AEF]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 3085830 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[E1005774C8280AEF]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001
   [junit4]   2> 3085830 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[E1005774C8280AEF]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3085830 INFO  (Thread-6752) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3085830 INFO  (Thread-6752) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3085832 ERROR (Thread-6752) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 3085930 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[E1005774C8280AEF]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:34975
   [junit4]   2> 3085932 INFO  (zkConnectionManagerCallback-7538-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3085935 INFO  (jetty-launcher-7535-thread-2) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 3085935 INFO  (jetty-launcher-7535-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 3085935 INFO  (jetty-launcher-7535-thread-3) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 3085936 INFO  (jetty-launcher-7535-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 3085936 INFO  (jetty-launcher-7535-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 3085936 INFO  (jetty-launcher-7535-thread-2) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 3085936 INFO  (jetty-launcher-7535-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 3085936 INFO  (jetty-launcher-7535-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 3085936 INFO  (jetty-launcher-7535-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 3085936 INFO  (jetty-launcher-7535-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 3085936 INFO  (jetty-launcher-7535-thread-3) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 3085936 INFO  (jetty-launcher-7535-thread-1) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 3085936 INFO  (jetty-launcher-7535-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4c727c30{/solr,null,AVAILABLE}
   [junit4]   2> 3085936 INFO  (jetty-launcher-7535-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3dbb051e{/solr,null,AVAILABLE}
   [junit4]   2> 3085936 INFO  (jetty-launcher-7535-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@64e8b9ff{/solr,null,AVAILABLE}
   [junit4]   2> 3085937 INFO  (jetty-launcher-7535-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@25ff48e3{SSL,[ssl, http/1.1]}{127.0.0.1:38775}
   [junit4]   2> 3085937 INFO  (jetty-launcher-7535-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4aed98b0{SSL,[ssl, http/1.1]}{127.0.0.1:41631}
   [junit4]   2> 3085937 INFO  (jetty-launcher-7535-thread-2) [    ] o.e.j.s.Server Started @3088100ms
   [junit4]   2> 3085937 INFO  (jetty-launcher-7535-thread-1) [    ] o.e.j.s.Server Started @3088100ms
   [junit4]   2> 3085937 INFO  (jetty-launcher-7535-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38775}
   [junit4]   2> 3085937 INFO  (jetty-launcher-7535-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41631}
   [junit4]   2> 3085937 INFO  (jetty-launcher-7535-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7b469701{SSL,[ssl, http/1.1]}{127.0.0.1:38349}
   [junit4]   2> 3085937 INFO  (jetty-launcher-7535-thread-3) [    ] o.e.j.s.Server Started @3088100ms
   [junit4]   2> 3085937 INFO  (jetty-launcher-7535-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38349}
   [junit4]   2> 3085937 ERROR (jetty-launcher-7535-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3085937 ERROR (jetty-launcher-7535-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3085938 INFO  (jetty-launcher-7535-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.3.0
   [junit4]   2> 3085938 INFO  (jetty-launcher-7535-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.3.0
   [junit4]   2> 3085938 INFO  (jetty-launcher-7535-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3085938 INFO  (jetty-launcher-7535-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3085938 INFO  (jetty-launcher-7535-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3085938 INFO  (jetty-launcher-7535-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3085938 INFO  (jetty-launcher-7535-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-23T03:02:27.436Z
   [junit4]   2> 3085938 INFO  (jetty-launcher-7535-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-23T03:02:27.436Z
   [junit4]   2> 3085938 ERROR (jetty-launcher-7535-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3085938 INFO  (jetty-launcher-7535-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.3.0
   [junit4]   2> 3085938 INFO  (jetty-launcher-7535-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3085938 INFO  (jetty-launcher-7535-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3085938 INFO  (jetty-launcher-7535-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-23T03:02:27.436Z
   [junit4]   2> 3085939 INFO  (zkConnectionManagerCallback-7542-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3085939 INFO  (zkConnectionManagerCallback-7541-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3085939 INFO  (zkConnectionManagerCallback-7544-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3085939 INFO  (jetty-launcher-7535-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3085939 INFO  (jetty-launcher-7535-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3085939 INFO  (jetty-launcher-7535-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3085944 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x16120f783010001, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 3085946 INFO  (jetty-launcher-7535-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34975/solr
   [junit4]   2> 3085951 INFO  (jetty-launcher-7535-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34975/solr
   [junit4]   2> 3085952 INFO  (zkConnectionManagerCallback-7549-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3085952 INFO  (jetty-launcher-7535-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34975/solr
   [junit4]   2> 3085952 INFO  (zkConnectionManagerCallback-7553-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3085952 INFO  (zkConnectionManagerCallback-7556-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3085953 INFO  (zkConnectionManagerCallback-7558-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3085954 INFO  (zkConnectionManagerCallback-7560-thread-1-processing-n:127.0.0.1:38775_solr) [n:127.0.0.1:38775_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3085954 INFO  (zkConnectionManagerCallback-7562-thread-1-processing-n:127.0.0.1:38349_solr) [n:127.0.0.1:38349_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3085995 INFO  (jetty-launcher-7535-thread-1) [n:127.0.0.1:41631_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3085995 INFO  (jetty-launcher-7535-thread-2) [n:127.0.0.1:38775_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3085995 INFO  (jetty-launcher-7535-thread-3) [n:127.0.0.1:38349_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3085995 INFO  (jetty-launcher-7535-thread-1) [n:127.0.0.1:41631_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41631_solr
   [junit4]   2> 3085995 INFO  (jetty-launcher-7535-thread-3) [n:127.0.0.1:38349_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38349_solr
   [junit4]   2> 3085995 INFO  (jetty-launcher-7535-thread-2) [n:127.0.0.1:38775_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38775_solr
   [junit4]   2> 3085995 INFO  (jetty-launcher-7535-thread-1) [n:127.0.0.1:41631_solr    ] o.a.s.c.Overseer Overseer (id=99396914205753351-127.0.0.1:41631_solr-n_0000000000) starting
   [junit4]   2> 3086006 INFO  (zkCallback-7559-thread-1-processing-n:127.0.0.1:38775_solr) [n:127.0.0.1:38775_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 3086007 INFO  (zkCallback-7557-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 3086007 INFO  (zkCallback-7561-thread-1-processing-n:127.0.0.1:38349_solr) [n:127.0.0.1:38349_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 3086009 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 3086009 DEBUG (OverseerAutoScalingTriggerThread-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 3086009 INFO  (jetty-launcher-7535-thread-1) [n:127.0.0.1:41631_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41631_solr
   [junit4]   2> 3086010 DEBUG (OverseerAutoScalingTriggerThread-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 3086010 INFO  (zkCallback-7561-thread-1-processing-n:127.0.0.1:38349_solr) [n:127.0.0.1:38349_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 3086010 INFO  (zkCallback-7559-thread-1-processing-n:127.0.0.1:38775_solr) [n:127.0.0.1:38775_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 3086029 INFO  (zkCallback-7557-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 3086029 DEBUG (OverseerAutoScalingTriggerThread-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:41631_solr, 127.0.0.1:38775_solr, 127.0.0.1:38349_solr]
   [junit4]   2> 3086029 DEBUG (OverseerAutoScalingTriggerThread-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 3086029 DEBUG (OverseerAutoScalingTriggerThread-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 3086034 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:41631_solr"} current state version: 0
   [junit4]   2> 3086034 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:41631_solr
   [junit4]   2> 3086036 DEBUG (OverseerAutoScalingTriggerThread-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3086036 DEBUG (ScheduledTrigger-7198-thread-1) [n:127.0.0.1:41631_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 3086036 DEBUG (OverseerAutoScalingTriggerThread-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 3086069 INFO  (jetty-launcher-7535-thread-3) [n:127.0.0.1:38349_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38349.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@722f4181
   [junit4]   2> 3086077 INFO  (jetty-launcher-7535-thread-3) [n:127.0.0.1:38349_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38349.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@722f4181
   [junit4]   2> 3086077 INFO  (jetty-launcher-7535-thread-3) [n:127.0.0.1:38349_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38349.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@722f4181
   [junit4]   2> 3086079 INFO  (jetty-launcher-7535-thread-3) [n:127.0.0.1:38349_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/.
   [junit4]   2> 3086101 INFO  (jetty-launcher-7535-thread-2) [n:127.0.0.1:38775_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38775.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@722f4181
   [junit4]   2> 3086113 INFO  (jetty-launcher-7535-thread-2) [n:127.0.0.1:38775_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38775.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@722f4181
   [junit4]   2> 3086113 INFO  (jetty-launcher-7535-thread-2) [n:127.0.0.1:38775_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38775.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@722f4181
   [junit4]   2> 3086114 INFO  (jetty-launcher-7535-thread-2) [n:127.0.0.1:38775_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/.
   [junit4]   2> 3086130 INFO  (jetty-launcher-7535-thread-1) [n:127.0.0.1:41631_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41631.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@722f4181
   [junit4]   2> 3086137 INFO  (jetty-launcher-7535-thread-1) [n:127.0.0.1:41631_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41631.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@722f4181
   [junit4]   2> 3086137 INFO  (jetty-launcher-7535-thread-1) [n:127.0.0.1:41631_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41631.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@722f4181
   [junit4]   2> 3086138 INFO  (jetty-launcher-7535-thread-1) [n:127.0.0.1:41631_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node1/.
   [junit4]   2> 3086202 INFO  (zkConnectionManagerCallback-7570-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3086214 INFO  (zkConnectionManagerCallback-7574-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3086214 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[E1005774C8280AEF]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 3086215 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[E1005774C8280AEF]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34975/solr ready
   [junit4]   2> 3086247 INFO  (qtp977896456-32486) [n:127.0.0.1:38775_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=3&name=AutoscalingHistoryHandlerTest_collection&nrtReplicas=3&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 3086250 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection AutoscalingHistoryHandlerTest_collection
   [junit4]   2> 3086250 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/protwords.txt
   [junit4]   2> 3086251 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/managed-schema
   [junit4]   2> 3086251 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/solrconfig.xml
   [junit4]   2> 3086252 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/synonyms.txt
   [junit4]   2> 3086253 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/stopwords.txt
   [junit4]   2> 3086253 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_it.txt
   [junit4]   2> 3086254 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stemdict_nl.txt
   [junit4]   2> 3086254 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hy.txt
   [junit4]   2> 3086255 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_no.txt
   [junit4]   2> 3086255 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ca.txt
   [junit4]   2> 3086256 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_id.txt
   [junit4]   2> 3086256 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ga.txt
   [junit4]   2> 3086257 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hi.txt
   [junit4]   2> 3086257 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_da.txt
   [junit4]   2> 3086258 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ja.txt
   [junit4]   2> 3086258 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_pt.txt
   [junit4]   2> 3086259 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hu.txt
   [junit4]   2> 3086259 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_el.txt
   [junit4]   2> 3086260 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ru.txt
   [junit4]   2> 3086260 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_tr.txt
   [junit4]   2> 3086261 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ar.txt
   [junit4]   2> 3086261 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/userdict_ja.txt
   [junit4]   2> 3086262 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_eu.txt
   [junit4]   2> 3086262 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_it.txt
   [junit4]   2> 3086263 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_cz.txt
   [junit4]   2> 3086263 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stoptags_ja.txt
   [junit4]   2> 3086266 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_fr.txt
   [junit4]   2> 3086267 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_de.txt
   [junit4]   2> 3086267 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fa.txt
   [junit4]   2> 3086268 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ga.txt
   [junit4]   2> 3086268 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ca.txt
   [junit4]   2> 3086269 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_nl.txt
   [junit4]   2> 3086269 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_bg.txt
   [junit4]   2> 3086270 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_lv.txt
   [junit4]   2> 3086271 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_en.txt
   [junit4]   2> 3086271 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_es.txt
   [junit4]   2> 3086272 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_th.txt
   [junit4]   2> 3086272 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ro.txt
   [junit4]   2> 3086273 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fr.txt
   [junit4]   2> 3086273 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_sv.txt
   [junit4]   2> 3086274 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fi.txt
   [junit4]   2> 3086274 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/hyphenations_ga.txt
   [junit4]   2> 3086275 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_gl.txt
   [junit4]   2> 3086275 INFO  (OverseerThreadFactory-7200-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/params.json to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/params.json
   [junit4]   2> 3086277 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "name":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 0
   [junit4]   2> 3086277 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.ClusterStateMutator building a new cName: AutoscalingHistoryHandlerTest_collection
   [junit4]   2> 3086277 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json
   [junit4]   2> 3086380 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:41631/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 3086380 INFO  (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:41631/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 3086380 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{}}
   [junit4]   2> 3086381 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:41631/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 3086381 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:38349/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 3086381 INFO  (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:38349/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 3086381 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:41631/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 3086382 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:41631/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>       "base_url":"https://127.0.0.1:38349/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 3086382 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:38775/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 3086382 INFO  (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:38775/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 3086382 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:41631/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>       "base_url":"https://127.0.0.1:38349/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 3086382 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:41631/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>       "base_url":"https://127.0.0.1:38349/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node6":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>       "base_url":"https://127.0.0.1:38775/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 3086483 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 0
   [junit4]   2> 3086586 INFO  (qtp977896456-32489) [n:127.0.0.1:38775_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n4&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 3086587 INFO  (qtp977896456-32489) [n:127.0.0.1:38775_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3086588 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "core_node_name":"core_node6",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:38775/solr",
   [junit4]   2>   "node_name":"127.0.0.1:38775_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 3086588 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "core_node_name":"core_node6",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:38775/solr",
   [junit4]   2>   "node_name":"127.0.0.1:38775_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 3086589 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node6:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4","base_url":"https://127.0.0.1:38775/solr","node_name":"127.0.0.1:38775_solr","state":"down","type":"NRT"}
   [junit4]   2> 3086589 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:41631/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:38349/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:38775/solr",
   [junit4]   2>           "node_name":"127.0.0.1:38775_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 3086606 INFO  (qtp1533112656-32496) [n:127.0.0.1:38349_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n2&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 3086607 INFO  (qtp1533112656-32496) [n:127.0.0.1:38349_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3086608 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node5",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:38349/solr",
   [junit4]   2>   "node_name":"127.0.0.1:38349_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 3086609 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node5",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:38349/solr",
   [junit4]   2>   "node_name":"127.0.0.1:38349_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 3086609 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node5:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2","base_url":"https://127.0.0.1:38349/solr","node_name":"127.0.0.1:38349_solr","state":"down","type":"NRT"}
   [junit4]   2> 3086609 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:41631/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:38349/solr",
   [junit4]   2>           "node_name":"127.0.0.1:38349_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:38775/solr",
   [junit4]   2>           "node_name":"127.0.0.1:38775_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 3086610 INFO  (qtp980342738-32497) [n:127.0.0.1:41631_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 3086611 INFO  (qtp980342738-32497) [n:127.0.0.1:41631_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3086612 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:41631/solr",
   [junit4]   2>   "node_name":"127.0.0.1:41631_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 3086612 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:41631/solr",
   [junit4]   2>   "node_name":"127.0.0.1:41631_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 3086612 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node3:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1","base_url":"https://127.0.0.1:41631/solr","node_name":"127.0.0.1:41631_solr","state":"down","type":"NRT"}
   [junit4]   2> 3086612 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:41631/solr",
   [junit4]   2>           "node_name":"127.0.0.1:41631_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:38349/solr",
   [junit4]   2>           "node_name":"127.0.0.1:38349_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:38775/solr",
   [junit4]   2>           "node_name":"127.0.0.1:38775_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 3086713 DEBUG (OverseerStateUpdate-99396914205753351-127.0.0.1:41631_solr-n_0000000000) [n:127.0.0.1:41631_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 1
   [junit4]   2> 3086714 INFO  (zkCallback-7557-thread-1-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 3086714 INFO  (zkCallback-7561-thread-1-processing-n:127.0.0.1:38349_solr) [n:127.0.0.1:38349_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 3086714 INFO  (zkCallback-7557-thread-2-processing-n:127.0.0.1:41631_solr) [n:127.0.0.1:41631_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 3086714 INFO  (zkCallback-7559-thread-2-processing-n:127.0.0.1:38775_solr) [n:127.0.0.1:38775_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 3086714 INFO  (zkCallback-7559-thread-1-processing-n:127.0.0.1:38775_solr) [n:127.0.0.1:38775_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 3086714 INFO  (zkCallback-7561-thread-2-processing-n:127.0.0.1:38349_solr) [n:127.0.0.1:38349_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 3087038 DEBUG (ScheduledTrigger-7198-thread-1) [n:127.0.0.1:41631_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 3087596 INFO  (qtp977896456-32489) [n:127.0.0.1:38775_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 3087596 INFO  (qtp977896456-32489) [n:127.0.0.1:38775_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 3087597 WARN  (qtp977896456-32489) [n:127.0.0.1:38775_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/extraction/lib
   [junit4]   2> 3087597 WARN  (qtp977896456-32489) [n:127.0.0.1:38775_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
   [junit4]   2> 3087597 WARN  (qtp977896456-32489) [n:127.0.0.1:38775_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/clustering/lib
   [junit4]   2> 3087597 WARN  (qtp977896456-32489) [n:127.0.0.1:38775_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
   [junit4]   2> 3087597 WARN  (qtp977896456-32489) [n:127.0.0.1:38775_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/langid/lib
   [junit4]   2> 3087597 WARN  (qtp977896456-32489) [n:127.0.0.1:38775_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
   [junit4]   2> 3087597 WARN  (qtp977896456-32489) [n:127.0.0.1:38775_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/velocity/lib
   [junit4]   2> 3087598 WARN  (qtp977896456-32489) [n:127.0.0.1:38775_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
   [junit4]   2> 3087603 INFO  (qtp977896456-32489) [n:127.0.0.1:38775_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 3087614 INFO  (qtp1533112656-32496) [n:127.0.0.1:38349_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 3087615 INFO  (qtp1533112656-32496) [n:127.0.0.1:38349_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 3087615 WARN  (qtp1533112656-32496) [n:127.0.0.1:38349_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib
   [junit4]   2> 3087616 WARN  (qtp1533112656-32496) [n:127.0.0.1:38349_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 3087616 WARN  (qtp1533112656-32496) [n:127.0.0.1:38349_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib
   [junit4]   2> 3087616 WARN  (qtp1533112656-32496) [n:127.0.0.1:38349_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 3087616 WARN  (qtp1533112656-32496) [n:127.0.0.1:38349_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib
   [junit4]   2> 3087617 WARN  (qtp1533112656-32496) [n:127.0.0.1:38349_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 3087617 WARN  (qtp1533112656-32496) [n:127.0.0.1:38349_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/velocity/lib
   [junit4]   2> 3087617 WARN  (qtp1533112656-32496) [n:127.0.0.1:38349_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 3087625 INFO  (qtp1533112656-32496) [n:127.0.0.1:38349_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 3087631 INFO  (qtp977896456-32489) [n:127.0.0.1:38775_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.s.IndexSchema [AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] Schema name=default-config
   [junit4]   2> 3087635 INFO  (qtp980342738-32497) [n:127.0.0.1:41631_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 3087678 INFO  (qtp980342738-32497) [n:127.0.0.1:41631_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 3087679 WARN  (qtp980342738-32497) [n:127.0.0.1:41631_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib
   [junit4]   2> 3087679 WARN  (qtp980342738-32497) [n:127.0.0.1:41631_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 3087679 WARN  (qtp980342738-32497) [n:127.0.0.1:41631_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib
   [junit4]   2> 3087679 WARN  (qtp980342738-32497) [n:127.0.0.1:41631_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 3087680 WARN  (qtp980342738-32497) [n:127.0.0.1:41631_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_E1005774C8280AEF-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/langid/lib
   [junit4]   2> 3087680 WARN  (qtp980342738-32497) [n:127.0.0.1:41631_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Cou

[...truncated too long message...]

 [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]   2>         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]   2>         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>    2) Thread[id=108, name=qtp245207537-108, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]   2>         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]   2>         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Jan 23, 2018 3:11:51 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=114, name=qtp245207537-114, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
   [junit4]   2>    2) Thread[id=108, name=qtp245207537-108, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
   [junit4]   2> Jan 23, 2018 3:11:54 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=114, name=qtp245207537-114, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]   2>         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]   2>         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>    2) Thread[id=108, name=qtp245207537-108, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]   2>         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]   2>         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {dbdesc_s=PostingsFormat(name=Asserting), dbid_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=PostingsFormat(name=Asserting), desc=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{}, maxPointsInLeafNode=652, maxMBSortInHeap=7.036331316852156, sim=RandomSimilarity(queryNorm=true): {}, locale=ar-LY, timezone=Asia/Chungking
   [junit4]   2> NOTE: Linux 4.13.0-26-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=8,threads=4,free=321336696,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [TestNumberFormatTransformer, TestTemplateTransformer, TestEntityProcessorBase, TestBuiltInEvaluators, TestErrorHandling, TestLineEntityProcessor, TestJdbcDataSourceConvertType, TestScriptTransformer, TestSolrEntityProcessorEndToEnd]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSolrEntityProcessorEndToEnd -Dtests.seed=EB24F3408972FCDB -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar-LY -Dtests.timezone=Asia/Chungking -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J1 | TestSolrEntityProcessorEndToEnd (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd:
   [junit4]    >    1) Thread[id=114, name=qtp245207537-114, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]    >         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]    >         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    2) Thread[id=108, name=qtp245207537-108, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]    >         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]    >         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([EB24F3408972FCDB]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=114, name=qtp245207537-114, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]    >         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]    >         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    2) Thread[id=108, name=qtp245207537-108, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]    >         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]    >         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([EB24F3408972FCDB]:0)
   [junit4] Completed [38/38 (1!)] on J1 in 62.60s, 7 tests, 2 errors <<< FAILURES!

[...truncated 42514 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-10-ea+37) - Build # 1226 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1226/
Java: 64bit/jdk-10-ea+37 -XX:+UseCompressedOops -XX:+UseSerialGC

2 tests failed.
FAILED:  org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.test

Error Message:
Error from server at https://127.0.0.1:34337/solr: no such collection or alias

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException: Error from server at https://127.0.0.1:34337/solr: no such collection or alias
        at __randomizedtesting.SeedInfo.seed([36F490D22F31B59F:BEA0AF0881CDD867]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException.create(HttpSolrClient.java:830)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:620)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
        at org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.test(TimeRoutedAliasUpdateProcessorTest.java:107)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory

Error Message:
expected:<5> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
        at __randomizedtesting.SeedInfo.seed([36F490D22F31B59F:5B08342F95794A98]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:265)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 12984 lines...]
   [junit4] Suite: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
   [junit4]   2> 1359221 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/init-core-data-001
   [junit4]   2> 1359221 WARN  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=55 numCloses=55
   [junit4]   2> 1359221 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1359223 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
   [junit4]   2> 1359224 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001
   [junit4]   2> 1359224 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1359237 INFO  (Thread-3824) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1359237 INFO  (Thread-3824) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1359240 ERROR (Thread-3824) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1359337 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:43631
   [junit4]   2> 1359338 INFO  (zkConnectionManagerCallback-2176-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359341 INFO  (jetty-launcher-2173-thread-2) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:57:37-03:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1359341 INFO  (jetty-launcher-2173-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:57:37-03:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1359341 INFO  (jetty-launcher-2173-thread-3) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:57:37-03:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1359362 INFO  (jetty-launcher-2173-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1359362 INFO  (jetty-launcher-2173-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1359362 INFO  (jetty-launcher-2173-thread-3) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1359365 INFO  (jetty-launcher-2173-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@108f3832{/solr,null,AVAILABLE}
   [junit4]   2> 1359366 INFO  (jetty-launcher-2173-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4a75de22{SSL,[ssl, http/1.1]}{127.0.0.1:?????}
   [junit4]   2> 1359366 INFO  (jetty-launcher-2173-thread-3) [    ] o.e.j.s.Server Started @???????ms
   [junit4]   2> 1359368 INFO  (jetty-launcher-2173-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41065}
   [junit4]   2> 1359368 ERROR (jetty-launcher-2173-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1359368 INFO  (jetty-launcher-2173-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.3.0
   [junit4]   2> 1359368 INFO  (jetty-launcher-2173-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1359368 INFO  (jetty-launcher-2173-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1359368 INFO  (jetty-launcher-2173-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-23T05:40:48.851607Z
   [junit4]   2> 1359402 INFO  (zkConnectionManagerCallback-2178-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359402 INFO  (jetty-launcher-2173-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1359444 INFO  (jetty-launcher-2173-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1359444 INFO  (jetty-launcher-2173-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1359444 INFO  (jetty-launcher-2173-thread-1) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1359445 INFO  (jetty-launcher-2173-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2e617bb4{/solr,null,AVAILABLE}
   [junit4]   2> 1359445 INFO  (jetty-launcher-2173-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3b32c7de{SSL,[ssl, http/1.1]}{127.0.0.1:?????}
   [junit4]   2> 1359445 INFO  (jetty-launcher-2173-thread-1) [    ] o.e.j.s.Server Started @???????ms
   [junit4]   2> 1359445 INFO  (jetty-launcher-2173-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39453}
   [junit4]   2> 1359446 ERROR (jetty-launcher-2173-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1359446 INFO  (jetty-launcher-2173-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.3.0
   [junit4]   2> 1359446 INFO  (jetty-launcher-2173-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1359446 INFO  (jetty-launcher-2173-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1359446 INFO  (jetty-launcher-2173-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-23T05:40:48.929351Z
   [junit4]   2> 1359447 INFO  (jetty-launcher-2173-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43631/solr
   [junit4]   2> 1359497 INFO  (jetty-launcher-2173-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1359497 INFO  (jetty-launcher-2173-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1359497 INFO  (jetty-launcher-2173-thread-2) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1359499 INFO  (zkConnectionManagerCallback-2181-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359499 INFO  (jetty-launcher-2173-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5f8e3d94{/solr,null,AVAILABLE}
   [junit4]   2> 1359499 INFO  (jetty-launcher-2173-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2faf2bd{SSL,[ssl, http/1.1]}{127.0.0.1:?????}
   [junit4]   2> 1359499 INFO  (jetty-launcher-2173-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1359500 INFO  (jetty-launcher-2173-thread-2) [    ] o.e.j.s.Server Started @???????ms
   [junit4]   2> 1359500 INFO  (jetty-launcher-2173-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35665}
   [junit4]   2> 1359500 ERROR (jetty-launcher-2173-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1359500 INFO  (jetty-launcher-2173-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.3.0
   [junit4]   2> 1359500 INFO  (jetty-launcher-2173-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1359500 INFO  (jetty-launcher-2173-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1359500 INFO  (jetty-launcher-2173-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-23T05:40:48.983649Z
   [junit4]   2> 1359507 INFO  (zkConnectionManagerCallback-2186-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359508 INFO  (jetty-launcher-2173-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43631/solr
   [junit4]   2> 1359508 INFO  (jetty-launcher-2173-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1359509 INFO  (zkConnectionManagerCallback-2184-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359517 INFO  (zkConnectionManagerCallback-2190-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359519 INFO  (jetty-launcher-2173-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43631/solr
   [junit4]   2> 1359549 INFO  (zkConnectionManagerCallback-2198-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359565 INFO  (zkConnectionManagerCallback-2192-thread-1-processing-n:127.0.0.1:41065_solr) [n:127.0.0.1:41065_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359601 INFO  (zkConnectionManagerCallback-2195-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359609 INFO  (jetty-launcher-2173-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1359609 INFO  (jetty-launcher-2173-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39453_solr
   [junit4]   2> 1359609 INFO  (jetty-launcher-2173-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer Overseer (id=99397536889045000-127.0.0.1:39453_solr-n_0000000000) starting
   [junit4]   2> 1359613 INFO  (zkConnectionManagerCallback-2200-thread-1-processing-n:127.0.0.1:35665_solr) [n:127.0.0.1:35665_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359617 INFO  (jetty-launcher-2173-thread-2) [n:127.0.0.1:35665_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1359618 INFO  (jetty-launcher-2173-thread-2) [n:127.0.0.1:35665_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35665_solr
   [junit4]   2> 1359633 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1359645 INFO  (jetty-launcher-2173-thread-3) [n:127.0.0.1:41065_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1359647 INFO  (jetty-launcher-2173-thread-3) [n:127.0.0.1:41065_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1359648 INFO  (jetty-launcher-2173-thread-3) [n:127.0.0.1:41065_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41065_solr
   [junit4]   2> 1359649 INFO  (jetty-launcher-2173-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39453_solr
   [junit4]   2> 1359668 DEBUG (OverseerAutoScalingTriggerThread-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 1359669 DEBUG (OverseerAutoScalingTriggerThread-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 1359669 DEBUG (OverseerAutoScalingTriggerThread-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: []
   [junit4]   2> 1359669 DEBUG (OverseerAutoScalingTriggerThread-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 1359669 DEBUG (OverseerAutoScalingTriggerThread-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 1359702 INFO  (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1359703 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:41065_solr"} current state version: 0
   [junit4]   2> 1359703 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:41065_solr
   [junit4]   2> 1359703 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:39453_solr"} current state version: 0
   [junit4]   2> 1359703 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:39453_solr
   [junit4]   2> 1359709 DEBUG (OverseerAutoScalingTriggerThread-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1359713 DEBUG (OverseerAutoScalingTriggerThread-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 1359714 INFO  (jetty-launcher-2173-thread-2) [n:127.0.0.1:35665_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35665.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359722 INFO  (zkCallback-2199-thread-1-processing-n:127.0.0.1:35665_solr) [n:127.0.0.1:35665_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1359722 INFO  (zkCallback-2191-thread-1-processing-n:127.0.0.1:41065_solr) [n:127.0.0.1:41065_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
   [junit4]   2> 1359722 INFO  (jetty-launcher-2173-thread-2) [n:127.0.0.1:35665_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35665.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359722 INFO  (jetty-launcher-2173-thread-2) [n:127.0.0.1:35665_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35665.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359724 INFO  (jetty-launcher-2173-thread-2) [n:127.0.0.1:35665_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/.
   [junit4]   2> 1359725 DEBUG (ScheduledTrigger-5897-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1359753 INFO  (jetty-launcher-2173-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39453.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359753 INFO  (jetty-launcher-2173-thread-3) [n:127.0.0.1:41065_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41065.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359761 INFO  (jetty-launcher-2173-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39453.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359761 INFO  (jetty-launcher-2173-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39453.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359761 INFO  (jetty-launcher-2173-thread-3) [n:127.0.0.1:41065_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41065.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359762 INFO  (jetty-launcher-2173-thread-3) [n:127.0.0.1:41065_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41065.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359762 INFO  (jetty-launcher-2173-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/.
   [junit4]   2> 1359762 INFO  (jetty-launcher-2173-thread-3) [n:127.0.0.1:41065_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/.
   [junit4]   2> 1359964 INFO  (zkConnectionManagerCallback-2208-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359969 INFO  (zkConnectionManagerCallback-2212-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359971 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1359972 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:43631/solr ready
   [junit4]   2> 1359992 INFO  (qtp1529629596-12812) [n:127.0.0.1:35665_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=3&name=AutoscalingHistoryHandlerTest_collection&nrtReplicas=3&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1359994 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection AutoscalingHistoryHandlerTest_collection
   [junit4]   2> 1359995 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/protwords.txt
   [junit4]   2> 1359996 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/managed-schema
   [junit4]   2> 1359997 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/solrconfig.xml
   [junit4]   2> 1359998 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/synonyms.txt
   [junit4]   2> 1359999 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/stopwords.txt
   [junit4]   2> 1359999 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_it.txt
   [junit4]   2> 1360001 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stemdict_nl.txt
   [junit4]   2> 1360002 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hy.txt
   [junit4]   2> 1360003 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_no.txt
   [junit4]   2> 1360004 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ca.txt
   [junit4]   2> 1360004 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_id.txt
   [junit4]   2> 1360005 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ga.txt
   [junit4]   2> 1360007 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hi.txt
   [junit4]   2> 1360008 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_da.txt
   [junit4]   2> 1360009 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ja.txt
   [junit4]   2> 1360011 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_pt.txt
   [junit4]   2> 1360011 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hu.txt
   [junit4]   2> 1360012 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_el.txt
   [junit4]   2> 1360012 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ru.txt
   [junit4]   2> 1360013 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_tr.txt
   [junit4]   2> 1360013 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ar.txt
   [junit4]   2> 1360014 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/userdict_ja.txt
   [junit4]   2> 1360014 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_eu.txt
   [junit4]   2> 1360015 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_it.txt
   [junit4]   2> 1360016 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_cz.txt
   [junit4]   2> 1360016 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stoptags_ja.txt
   [junit4]   2> 1360017 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_fr.txt
   [junit4]   2> 1360017 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_de.txt
   [junit4]   2> 1360018 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fa.txt
   [junit4]   2> 1360019 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ga.txt
   [junit4]   2> 1360019 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ca.txt
   [junit4]   2> 1360020 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_nl.txt
   [junit4]   2> 1360021 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_bg.txt
   [junit4]   2> 1360023 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_lv.txt
   [junit4]   2> 1360024 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_en.txt
   [junit4]   2> 1360026 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_es.txt
   [junit4]   2> 1360027 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_th.txt
   [junit4]   2> 1360027 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ro.txt
   [junit4]   2> 1360029 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fr.txt
   [junit4]   2> 1360030 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_sv.txt
   [junit4]   2> 1360030 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fi.txt
   [junit4]   2> 1360031 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/hyphenations_ga.txt
   [junit4]   2> 1360032 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_gl.txt
   [junit4]   2> 1360032 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/params.json to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/params.json
   [junit4]   2> 1360037 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "name":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 0
   [junit4]   2> 1360037 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ClusterStateMutator building a new cName: AutoscalingHistoryHandlerTest_collection
   [junit4]   2> 1360037 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json
   [junit4]   2> 1360137 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 1360137 INFO  (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1360137 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{}}
   [junit4]   2> 1360137 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 1360138 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 1360138 INFO  (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1360138 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 1360138 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>       "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 1360139 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:41065/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 1360139 INFO  (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:41065/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1360140 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>       "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 1360140 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>       "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node6":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>       "base_url":"https://127.0.0.1:41065/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 1360240 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 0
   [junit4]   2> 1360385 INFO  (qtp1529629596-12795) [n:127.0.0.1:35665_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1360385 INFO  (qtp1529629596-12795) [n:127.0.0.1:35665_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1360400 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>   "node_name":"127.0.0.1:35665_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1360400 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>   "node_name":"127.0.0.1:35665_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1360400 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node3:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1","base_url":"https://127.0.0.1:35665/solr","node_name":"127.0.0.1:35665_solr","state":"down","type":"NRT"}
   [junit4]   2> 1360400 INFO  (qtp1965882153-12790) [n:127.0.0.1:41065_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n4&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1360400 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>           "node_name":"127.0.0.1:35665_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:41065/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 1360401 INFO  (qtp1762186773-12780) [n:127.0.0.1:39453_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n2&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1360401 INFO  (qtp1965882153-12790) [n:127.0.0.1:41065_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1360401 INFO  (qtp1762186773-12780) [n:127.0.0.1:39453_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1360405 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "core_node_name":"core_node6",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:41065/solr",
   [junit4]   2>   "node_name":"127.0.0.1:41065_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1360405 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "core_node_name":"core_node6",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:41065/solr",
   [junit4]   2>   "node_name":"127.0.0.1:41065_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1360405 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node6:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4","base_url":"https://127.0.0.1:41065/solr","node_name":"127.0.0.1:41065_solr","state":"down","type":"NRT"}
   [junit4]   2> 1360405 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>           "node_name":"127.0.0.1:35665_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:41065/solr",
   [junit4]   2>           "node_name":"127.0.0.1:41065_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 1360406 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node5",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>   "node_name":"127.0.0.1:39453_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1360406 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node5",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>   "node_name":"127.0.0.1:39453_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1360406 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node5:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2","base_url":"https://127.0.0.1:39453/solr","node_name":"127.0.0.1:39453_solr","state":"down","type":"NRT"}
   [junit4]   2> 1360406 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>           "node_name":"127.0.0.1:35665_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>           "node_name":"127.0.0.1:39453_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:41065/solr",
   [junit4]   2>           "node_name":"127.0.0.1:41065_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 1360506 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 1
   [junit4]   2> 1360506 INFO  (zkCallback-2191-thread-1-processing-n:127.0.0.1:41065_solr) [n:127.0.0.1:41065_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1360506 INFO  (zkCallback-2194-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1360507 INFO  (zkCallback-2199-thread-1-processing-n:127.0.0.1:35665_solr) [n:127.0.0.1:35665_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1360507 INFO  (zkCallback-2199-thread-2-processing-n:127.0.0.1:35665_solr) [n:127.0.0.1:35665_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1360509 INFO  (zkCallback-2194-thread-2-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1360513 INFO  (zkCallback-2191-thread-2-processing-n:127.0.0.1:41065_solr) [n:127.0.0.1:41065_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1360726 DEBUG (ScheduledTrigger-5897-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1361404 INFO  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 1361404 INFO  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 1361404 WARN  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib
   [junit4]   2> 1361404 WARN  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 1361404 WARN  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib
   [junit4]   2> 1361405 WARN  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 1361405 WARN  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/langid/lib
   [junit4]   2> 1361405 WARN  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 1361405 WARN  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/velocity/lib
   [junit4]   2> 1361405 WARN  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 1361409 INFO  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 1361409 INFO  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 1361409 WARN  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib
   [junit4]   2> 1361409 INFO  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 1361409 WARN  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 1361409 INFO  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 1361410 WARN  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib
   [junit4]   2> 1361410 WARN  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 1361410 WARN  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib
   [junit4]   2> 1361410 WARN  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/extraction/lib
   [junit4]   2> 1361410 WARN  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 1361410 WARN  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
   [junit4]   2> 1361410 INFO  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1361410 WARN  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/velocity/lib
   [junit4]   2> 1361410 WARN  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/clustering/lib
   [junit4]   2> 1361410 WARN  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 1361411 WARN  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
   [junit4]   2> 1361411 WARN  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/langid/lib
   [junit4]   2> 1361411 WARN  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
   [junit4]   2> 1361411 WARN  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/velocity/lib
   [junit4]   2> 1361411 WARN  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
   [junit4]   2> 1361416 INFO  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1361418 INFO  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1361430 INFO  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.s.IndexSchema [AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] Schema name=default-config
   [junit4]   2> 1361452 INFO  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.s.IndexSchema [AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] Schema name=default-config
   [junit4]   2> 1361454 INFO  (

[...truncated too long message...]

 Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:34337_solr
   [junit4]   2> 2552229 INFO  (jetty-closer-6632-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 2552229 INFO  (zkCallback-6649-thread-1-processing-n:127.0.0.1:34337_solr) [n:127.0.0.1:34337_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 2552229 INFO  (zkCallback-6647-thread-1-processing-n:127.0.0.1:44761_solr) [n:127.0.0.1:44761_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 2552229 INFO  (jetty-closer-6632-thread-2) [    ] o.a.s.c.Overseer Overseer (id=99397612939509766-127.0.0.1:34337_solr-n_0000000000) closing
   [junit4]   2> 2552229 INFO  (OverseerStateUpdate-99397612939509766-127.0.0.1:34337_solr-n_0000000000) [n:127.0.0.1:34337_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:34337_solr
   [junit4]   2> 2552230 WARN  (OverseerAutoScalingTriggerThread-99397612939509766-127.0.0.1:34337_solr-n_0000000000) [n:127.0.0.1:34337_solr    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 2552231 INFO  (jetty-closer-6632-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@621bf1c4{/solr,null,UNAVAILABLE}
   [junit4]   2> 2552231 INFO  (zkCallback-6647-thread-1-processing-n:127.0.0.1:44761_solr) [n:127.0.0.1:44761_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44761_solr
   [junit4]   2> 2552231 INFO  (jetty-closer-6632-thread-2) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 2552234 INFO  (jetty-closer-6632-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 2552234 INFO  (jetty-closer-6632-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@75dd2cdd: rootName = solr_44761, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@38047dbd
   [junit4]   2> 2552235 INFO  (jetty-closer-6632-thread-1) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:44761_solr
   [junit4]   2> 2552235 INFO  (jetty-closer-6632-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 2552236 INFO  (zkCallback-6647-thread-1-processing-n:127.0.0.1:44761_solr) [n:127.0.0.1:44761_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 2552236 INFO  (jetty-closer-6632-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2552237 INFO  (jetty-closer-6632-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@111ef513{/solr,null,UNAVAILABLE}
   [junit4]   2> 2552237 INFO  (jetty-closer-6632-thread-1) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 2552237 ERROR (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2552238 INFO  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44065 44065
   [junit4]   2> 2552826 INFO  (Thread-8551) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44065 44065
   [junit4]   2> 2552827 WARN  (Thread-8551) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/aliases.json
   [junit4]   2> 2 /solr/security.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/clusterstate.json
   [junit4]   2> 3 /solr/collections/timeConfig/state.json
   [junit4]   2> 3 /solr/clusterprops.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/live_nodes
   [junit4]   2> 3 /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_36F490D22F31B59F-001
   [junit4]   2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=true): {}, locale=saq, timezone=Asia/Saigon
   [junit4]   2> NOTE: Linux 4.13.0-26-generic amd64/Oracle Corporation 10-ea (64-bit)/cpus=8,threads=1,free=161175608,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [TestPolicyCloud, CleanupOldIndexTest, TestFaceting, MinimalSchemaTest, TestSchemalessBufferedUpdates, TestBlendedInfixSuggestions, TestFiltering, ReturnFieldsTest, SolrCmdDistributorTest, AutoAddReplicasPlanActionTest, CdcrVersionReplicationTest, RegexBoostProcessorTest, DistributedFacetPivotWhiteBoxTest, TestSolrCoreSnapshots, ClassificationUpdateProcessorFactoryTest, TestReplicaProperties, ShardSplitTest, TestTrackingShardHandlerFactory, TestXmlQParserPlugin, DistribJoinFromCollectionTest, TestScoreJoinQPScore, TestFastWriter, TestSSLRandomization, TestCryptoKeys, DistributedSuggestComponentTest, ManagedSchemaRoundRobinCloudTest, AnalysisAfterCoreReloadTest, DisMaxRequestHandlerTest, SolrMetricReporterTest, WordBreakSolrSpellCheckerTest, TestMacros, LeaderElectionContextKeyTest, TestHighFrequencyDictionaryFactory, SolrGraphiteReporterTest, TestConfigSetsAPI, TestSolrCoreProperties, TemplateUpdateProcessorTest, TestMiniSolrCloudClusterSSL, CoreAdminCreateDiscoverTest, TestGraphTermsQParserPlugin, CdcrReplicationDistributedZkTest, ReplicationFactorTest, TestSolrDeletionPolicy2, TestClusterStateMutator, TestPushWriter, TestAnalyzedSuggestions, TestQuerySenderListener, RecoveryAfterSoftCommitTest, URLClassifyProcessorTest, TestInfoStreamLogging, InfoHandlerTest, TestGeoJSONResponseWriter, TestMultiValuedNumericRangeQuery, TestPKIAuthenticationPlugin, TestJsonFacets, TestBulkSchemaAPI, ReplicaListTransformerTest, TestReloadAndDeleteDocs, TestValueSourceCache, TestLazyCores, ConfigureRecoveryStrategyTest, ClusterStateTest, BooleanFieldTest, AnalyticsMergeStrategyTest, TestRestoreCore, RecoveryZkTest, StatelessScriptUpdateProcessorFactoryTest, OverseerCollectionConfigSetProcessorTest, BasicDistributedZk2Test, TestDownShardTolerantSearch, TestStressReorder, PingRequestHandlerTest, TestInPlaceUpdatesStandalone, CreateCollectionCleanupTest, TestDistribStateManager, TriggerIntegrationTest, SimpleMLTQParserTest, SolrCLIZkUtilsTest, NodeMutatorTest, NoCacheHeaderTest, TestLFUCache, DeleteStatusTest, IndexSchemaRuntimeFieldTest, TestLeaderElectionZkExpiry, TestSolrConfigHandler, AliasIntegrationTest, LeaderInitiatedRecoveryOnCommitTest, ClusterStateUpdateTest, TestDocTermOrds, MetricTriggerTest, TestMultiWordSynonyms, WrapperMergePolicyFactoryTest, TestAtomicUpdateErrorCases, ExitableDirectoryReaderTest, TestDelegationWithHadoopAuth, HdfsUnloadDistributedZkTest, TestStressInPlaceUpdates, ImplicitSnitchTest, MoveReplicaHDFSTest, TestRandomCollapseQParserPlugin, TestCodecSupport, DistributedSpellCheckComponentTest, MultiTermTest, TestBlobHandler, TimeZoneUtilsTest, CopyFieldTest, TestNumericTerms64, V2ApiIntegrationTest, TestCloudJSONFacetJoinDomain, ExecutePlanActionTest, TestExpandComponent, CollectionTooManyReplicasTest, SpellCheckComponentTest, SearchHandlerTest, DocValuesNotIndexedTest, TestUtils, TestBadConfig, DirectSolrSpellCheckerTest, TestSortByMinMaxFunction, DistanceFunctionTest, SparseHLLTest, TestPayloadCheckQParserPlugin, TestUpdate, TestDFRSimilarityFactory, BaseCdcrDistributedZkTest, MoreLikeThisHandlerTest, TestQueryTypes, ChaosMonkeyNothingIsSafeWithPullReplicasTest, LeaderInitiatedRecoveryOnShardRestartTest, MigrateRouteKeyTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerModifyCollectionTest, PeerSyncReplicationTest, TestCloudSearcherWarming, TestRebalanceLeaders, TestSizeLimitedDistributedMap, TestSkipOverseerOperations, TestTolerantUpdateProcessorCloud, VMParamsZkACLAndCredentialsProvidersTest, CollectionsAPIAsyncDistributedZkTest, TestLocalFSCloudBackupRestore, NodeAddedTriggerTest, SystemLogListenerTest, TestComputePlanAction, TestLargeCluster, HdfsRecoveryZkTest, HdfsTlogReplayBufferedWhileIndexingTest, StressHdfsTest, TestInitParams, TestShardHandlerFactory, TestSolrCloudSnapshots, TestSQLHandler, PropertiesRequestHandlerTest, SegmentsInfoRequestHandlerTest, TestConfigsApi, CustomHighlightComponentTest, DistributedQueryComponentOptimizationTest, SolrMetricsIntegrationTest, SolrJmxReporterTest, SolrSlf4jReporterTest, SolrShardReporterTest, RegexBytesRefFilterTest, SubstringBytesRefFilterTest, TestIntervalFaceting, SpatialRPTFieldTypeTest, TestCloudManagedSchema, CurrencyRangeFacetCloudTest, TestComplexPhraseQParserPlugin, TestCustomSort, TestFieldSortValues, TestInitQParser, TestMissingGroups, TestSearcherReuse, TestSimpleQParserPlugin, TestSmileRequest, TestSolrCoreParser, TestStressUserVersions, TestTrieFacet, TestJsonFacetRefinement, TestMinMaxOnMultiValuedField, GraphQueryTest, TestNestedDocsSort, TestJsonRequest, CloudMLTQParserTest, TestDFISimilarityFactory, TestDefaultStatsCache, TestExactStatsCache, TestZkAclsWithHadoopAuth, ConjunctionSolrSpellCheckerTest, SpellCheckCollatorWithCollapseTest, BlockDirectoryTest, BufferStoreTest, TestFieldCache, TestFieldCacheSort, TestFieldCacheWithThreads, TestUninvertingReader, AnalysisErrorHandlingTest, PeerSyncWithIndexFingerprintCachingTest, AtomicUpdateProcessorFactoryTest, CloneFieldUpdateProcessorFactoryTest, PreAnalyzedUpdateProcessorTest, TimeRoutedAliasUpdateProcessorTest]
   [junit4] Completed [760/767 (2!)] on J2 in 33.09s, 2 tests, 1 error <<< FAILURES!

[...truncated 40910 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-10-ea+37) - Build # 1226 - Still Unstable!

Ishan Chattopadhyaya
I'll take a look. Maybe the commit for SOLR-11624 broke it.

On Tue, Jan 23, 2018 at 11:43 AM, Policeman Jenkins Server <[hidden email]> wrote:
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1226/
Java: 64bit/jdk-10-ea+37 -XX:+UseCompressedOops -XX:+UseSerialGC

2 tests failed.
FAILED:  org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.test

Error Message:
Error from server at https://127.0.0.1:34337/solr: no such collection or alias

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException: Error from server at https://127.0.0.1:34337/solr: no such collection or alias
        at __randomizedtesting.SeedInfo.seed([36F490D22F31B59F:BEA0AF0881CDD867]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException.create(HttpSolrClient.java:830)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:620)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
        at org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.test(TimeRoutedAliasUpdateProcessorTest.java:107)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory

Error Message:
expected:<5> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
        at __randomizedtesting.SeedInfo.seed([36F490D22F31B59F:5B08342F95794A98]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:265)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 12984 lines...]
   [junit4] Suite: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
   [junit4]   2> 1359221 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/init-core-data-001
   [junit4]   2> 1359221 WARN  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=55 numCloses=55
   [junit4]   2> 1359221 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1359223 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
   [junit4]   2> 1359224 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001
   [junit4]   2> 1359224 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1359237 INFO  (Thread-3824) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1359237 INFO  (Thread-3824) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1359240 ERROR (Thread-3824) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1359337 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:43631
   [junit4]   2> 1359338 INFO  (zkConnectionManagerCallback-2176-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359341 INFO  (jetty-launcher-2173-thread-2) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:57:37-03:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1359341 INFO  (jetty-launcher-2173-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:57:37-03:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1359341 INFO  (jetty-launcher-2173-thread-3) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:57:37-03:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1359362 INFO  (jetty-launcher-2173-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1359362 INFO  (jetty-launcher-2173-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1359362 INFO  (jetty-launcher-2173-thread-3) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1359365 INFO  (jetty-launcher-2173-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@108f3832{/solr,null,AVAILABLE}
   [junit4]   2> 1359366 INFO  (jetty-launcher-2173-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4a75de22{SSL,[ssl, http/1.1]}{127.0.0.1:?????}
   [junit4]   2> 1359366 INFO  (jetty-launcher-2173-thread-3) [    ] o.e.j.s.Server Started @???????ms
   [junit4]   2> 1359368 INFO  (jetty-launcher-2173-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41065}
   [junit4]   2> 1359368 ERROR (jetty-launcher-2173-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1359368 INFO  (jetty-launcher-2173-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.3.0
   [junit4]   2> 1359368 INFO  (jetty-launcher-2173-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1359368 INFO  (jetty-launcher-2173-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1359368 INFO  (jetty-launcher-2173-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-23T05:40:48.851607Z
   [junit4]   2> 1359402 INFO  (zkConnectionManagerCallback-2178-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359402 INFO  (jetty-launcher-2173-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1359444 INFO  (jetty-launcher-2173-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1359444 INFO  (jetty-launcher-2173-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1359444 INFO  (jetty-launcher-2173-thread-1) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1359445 INFO  (jetty-launcher-2173-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2e617bb4{/solr,null,AVAILABLE}
   [junit4]   2> 1359445 INFO  (jetty-launcher-2173-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3b32c7de{SSL,[ssl, http/1.1]}{127.0.0.1:?????}
   [junit4]   2> 1359445 INFO  (jetty-launcher-2173-thread-1) [    ] o.e.j.s.Server Started @???????ms
   [junit4]   2> 1359445 INFO  (jetty-launcher-2173-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39453}
   [junit4]   2> 1359446 ERROR (jetty-launcher-2173-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1359446 INFO  (jetty-launcher-2173-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.3.0
   [junit4]   2> 1359446 INFO  (jetty-launcher-2173-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1359446 INFO  (jetty-launcher-2173-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1359446 INFO  (jetty-launcher-2173-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-23T05:40:48.929351Z
   [junit4]   2> 1359447 INFO  (jetty-launcher-2173-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43631/solr
   [junit4]   2> 1359497 INFO  (jetty-launcher-2173-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1359497 INFO  (jetty-launcher-2173-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1359497 INFO  (jetty-launcher-2173-thread-2) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1359499 INFO  (zkConnectionManagerCallback-2181-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359499 INFO  (jetty-launcher-2173-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5f8e3d94{/solr,null,AVAILABLE}
   [junit4]   2> 1359499 INFO  (jetty-launcher-2173-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2faf2bd{SSL,[ssl, http/1.1]}{127.0.0.1:?????}
   [junit4]   2> 1359499 INFO  (jetty-launcher-2173-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1359500 INFO  (jetty-launcher-2173-thread-2) [    ] o.e.j.s.Server Started @???????ms
   [junit4]   2> 1359500 INFO  (jetty-launcher-2173-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35665}
   [junit4]   2> 1359500 ERROR (jetty-launcher-2173-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1359500 INFO  (jetty-launcher-2173-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.3.0
   [junit4]   2> 1359500 INFO  (jetty-launcher-2173-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1359500 INFO  (jetty-launcher-2173-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1359500 INFO  (jetty-launcher-2173-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-23T05:40:48.983649Z
   [junit4]   2> 1359507 INFO  (zkConnectionManagerCallback-2186-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359508 INFO  (jetty-launcher-2173-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43631/solr
   [junit4]   2> 1359508 INFO  (jetty-launcher-2173-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1359509 INFO  (zkConnectionManagerCallback-2184-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359517 INFO  (zkConnectionManagerCallback-2190-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359519 INFO  (jetty-launcher-2173-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43631/solr
   [junit4]   2> 1359549 INFO  (zkConnectionManagerCallback-2198-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359565 INFO  (zkConnectionManagerCallback-2192-thread-1-processing-n:127.0.0.1:41065_solr) [n:127.0.0.1:41065_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359601 INFO  (zkConnectionManagerCallback-2195-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359609 INFO  (jetty-launcher-2173-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1359609 INFO  (jetty-launcher-2173-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39453_solr
   [junit4]   2> 1359609 INFO  (jetty-launcher-2173-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer Overseer (id=99397536889045000-127.0.0.1:39453_solr-n_0000000000) starting
   [junit4]   2> 1359613 INFO  (zkConnectionManagerCallback-2200-thread-1-processing-n:127.0.0.1:35665_solr) [n:127.0.0.1:35665_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359617 INFO  (jetty-launcher-2173-thread-2) [n:127.0.0.1:35665_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1359618 INFO  (jetty-launcher-2173-thread-2) [n:127.0.0.1:35665_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35665_solr
   [junit4]   2> 1359633 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1359645 INFO  (jetty-launcher-2173-thread-3) [n:127.0.0.1:41065_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1359647 INFO  (jetty-launcher-2173-thread-3) [n:127.0.0.1:41065_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1359648 INFO  (jetty-launcher-2173-thread-3) [n:127.0.0.1:41065_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41065_solr
   [junit4]   2> 1359649 INFO  (jetty-launcher-2173-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39453_solr
   [junit4]   2> 1359668 DEBUG (OverseerAutoScalingTriggerThread-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 1359669 DEBUG (OverseerAutoScalingTriggerThread-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 1359669 DEBUG (OverseerAutoScalingTriggerThread-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: []
   [junit4]   2> 1359669 DEBUG (OverseerAutoScalingTriggerThread-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 1359669 DEBUG (OverseerAutoScalingTriggerThread-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 1359702 INFO  (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1359703 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:41065_solr"} current state version: 0
   [junit4]   2> 1359703 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:41065_solr
   [junit4]   2> 1359703 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:39453_solr"} current state version: 0
   [junit4]   2> 1359703 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:39453_solr
   [junit4]   2> 1359709 DEBUG (OverseerAutoScalingTriggerThread-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1359713 DEBUG (OverseerAutoScalingTriggerThread-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 1359714 INFO  (jetty-launcher-2173-thread-2) [n:127.0.0.1:35665_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35665.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359722 INFO  (zkCallback-2199-thread-1-processing-n:127.0.0.1:35665_solr) [n:127.0.0.1:35665_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1359722 INFO  (zkCallback-2191-thread-1-processing-n:127.0.0.1:41065_solr) [n:127.0.0.1:41065_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
   [junit4]   2> 1359722 INFO  (jetty-launcher-2173-thread-2) [n:127.0.0.1:35665_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35665.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359722 INFO  (jetty-launcher-2173-thread-2) [n:127.0.0.1:35665_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35665.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359724 INFO  (jetty-launcher-2173-thread-2) [n:127.0.0.1:35665_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/.
   [junit4]   2> 1359725 DEBUG (ScheduledTrigger-5897-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1359753 INFO  (jetty-launcher-2173-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39453.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359753 INFO  (jetty-launcher-2173-thread-3) [n:127.0.0.1:41065_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41065.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359761 INFO  (jetty-launcher-2173-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39453.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359761 INFO  (jetty-launcher-2173-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39453.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359761 INFO  (jetty-launcher-2173-thread-3) [n:127.0.0.1:41065_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41065.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359762 INFO  (jetty-launcher-2173-thread-3) [n:127.0.0.1:41065_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41065.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42b33169
   [junit4]   2> 1359762 INFO  (jetty-launcher-2173-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/.
   [junit4]   2> 1359762 INFO  (jetty-launcher-2173-thread-3) [n:127.0.0.1:41065_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/.
   [junit4]   2> 1359964 INFO  (zkConnectionManagerCallback-2208-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359969 INFO  (zkConnectionManagerCallback-2212-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1359971 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1359972 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:43631/solr ready
   [junit4]   2> 1359992 INFO  (qtp1529629596-12812) [n:127.0.0.1:35665_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=3&name=AutoscalingHistoryHandlerTest_collection&nrtReplicas=3&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1359994 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection AutoscalingHistoryHandlerTest_collection
   [junit4]   2> 1359995 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/protwords.txt
   [junit4]   2> 1359996 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/managed-schema
   [junit4]   2> 1359997 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/solrconfig.xml
   [junit4]   2> 1359998 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/synonyms.txt
   [junit4]   2> 1359999 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/stopwords.txt
   [junit4]   2> 1359999 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_it.txt
   [junit4]   2> 1360001 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stemdict_nl.txt
   [junit4]   2> 1360002 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hy.txt
   [junit4]   2> 1360003 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_no.txt
   [junit4]   2> 1360004 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ca.txt
   [junit4]   2> 1360004 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_id.txt
   [junit4]   2> 1360005 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ga.txt
   [junit4]   2> 1360007 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hi.txt
   [junit4]   2> 1360008 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_da.txt
   [junit4]   2> 1360009 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ja.txt
   [junit4]   2> 1360011 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_pt.txt
   [junit4]   2> 1360011 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hu.txt
   [junit4]   2> 1360012 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_el.txt
   [junit4]   2> 1360012 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ru.txt
   [junit4]   2> 1360013 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_tr.txt
   [junit4]   2> 1360013 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ar.txt
   [junit4]   2> 1360014 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/userdict_ja.txt
   [junit4]   2> 1360014 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_eu.txt
   [junit4]   2> 1360015 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_it.txt
   [junit4]   2> 1360016 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_cz.txt
   [junit4]   2> 1360016 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stoptags_ja.txt
   [junit4]   2> 1360017 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_fr.txt
   [junit4]   2> 1360017 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_de.txt
   [junit4]   2> 1360018 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fa.txt
   [junit4]   2> 1360019 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ga.txt
   [junit4]   2> 1360019 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ca.txt
   [junit4]   2> 1360020 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_nl.txt
   [junit4]   2> 1360021 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_bg.txt
   [junit4]   2> 1360023 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_lv.txt
   [junit4]   2> 1360024 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_en.txt
   [junit4]   2> 1360026 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_es.txt
   [junit4]   2> 1360027 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_th.txt
   [junit4]   2> 1360027 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ro.txt
   [junit4]   2> 1360029 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fr.txt
   [junit4]   2> 1360030 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_sv.txt
   [junit4]   2> 1360030 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fi.txt
   [junit4]   2> 1360031 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/hyphenations_ga.txt
   [junit4]   2> 1360032 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_gl.txt
   [junit4]   2> 1360032 INFO  (OverseerThreadFactory-5899-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/params.json to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/params.json
   [junit4]   2> 1360037 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "name":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 0
   [junit4]   2> 1360037 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ClusterStateMutator building a new cName: AutoscalingHistoryHandlerTest_collection
   [junit4]   2> 1360037 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json
   [junit4]   2> 1360137 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 1360137 INFO  (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1360137 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{}}
   [junit4]   2> 1360137 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 1360138 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 1360138 INFO  (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1360138 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 1360138 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>       "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 1360139 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:41065/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 1360139 INFO  (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:41065/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1360140 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>       "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 1360140 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>       "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node6":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>       "base_url":"https://127.0.0.1:41065/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 1360240 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 0
   [junit4]   2> 1360385 INFO  (qtp1529629596-12795) [n:127.0.0.1:35665_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1360385 INFO  (qtp1529629596-12795) [n:127.0.0.1:35665_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1360400 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>   "node_name":"127.0.0.1:35665_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1360400 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>   "node_name":"127.0.0.1:35665_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1360400 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node3:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1","base_url":"https://127.0.0.1:35665/solr","node_name":"127.0.0.1:35665_solr","state":"down","type":"NRT"}
   [junit4]   2> 1360400 INFO  (qtp1965882153-12790) [n:127.0.0.1:41065_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n4&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1360400 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>           "node_name":"127.0.0.1:35665_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:41065/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 1360401 INFO  (qtp1762186773-12780) [n:127.0.0.1:39453_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n2&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1360401 INFO  (qtp1965882153-12790) [n:127.0.0.1:41065_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1360401 INFO  (qtp1762186773-12780) [n:127.0.0.1:39453_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1360405 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "core_node_name":"core_node6",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:41065/solr",
   [junit4]   2>   "node_name":"127.0.0.1:41065_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1360405 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "core_node_name":"core_node6",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:41065/solr",
   [junit4]   2>   "node_name":"127.0.0.1:41065_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1360405 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node6:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4","base_url":"https://127.0.0.1:41065/solr","node_name":"127.0.0.1:41065_solr","state":"down","type":"NRT"}
   [junit4]   2> 1360405 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>           "node_name":"127.0.0.1:35665_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:41065/solr",
   [junit4]   2>           "node_name":"127.0.0.1:41065_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 1360406 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node5",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>   "node_name":"127.0.0.1:39453_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1360406 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node5",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>   "node_name":"127.0.0.1:39453_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1360406 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node5:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2","base_url":"https://127.0.0.1:39453/solr","node_name":"127.0.0.1:39453_solr","state":"down","type":"NRT"}
   [junit4]   2> 1360406 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:35665/solr",
   [junit4]   2>           "node_name":"127.0.0.1:35665_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:39453/solr",
   [junit4]   2>           "node_name":"127.0.0.1:39453_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>           "base_url":"https://127.0.0.1:41065/solr",
   [junit4]   2>           "node_name":"127.0.0.1:41065_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 1360506 DEBUG (OverseerStateUpdate-99397536889045000-127.0.0.1:39453_solr-n_0000000000) [n:127.0.0.1:39453_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 1
   [junit4]   2> 1360506 INFO  (zkCallback-2191-thread-1-processing-n:127.0.0.1:41065_solr) [n:127.0.0.1:41065_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1360506 INFO  (zkCallback-2194-thread-1-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1360507 INFO  (zkCallback-2199-thread-1-processing-n:127.0.0.1:35665_solr) [n:127.0.0.1:35665_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1360507 INFO  (zkCallback-2199-thread-2-processing-n:127.0.0.1:35665_solr) [n:127.0.0.1:35665_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1360509 INFO  (zkCallback-2194-thread-2-processing-n:127.0.0.1:39453_solr) [n:127.0.0.1:39453_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1360513 INFO  (zkCallback-2191-thread-2-processing-n:127.0.0.1:41065_solr) [n:127.0.0.1:41065_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 1360726 DEBUG (ScheduledTrigger-5897-thread-1) [n:127.0.0.1:39453_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1361404 INFO  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 1361404 INFO  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 1361404 WARN  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib
   [junit4]   2> 1361404 WARN  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 1361404 WARN  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib
   [junit4]   2> 1361405 WARN  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 1361405 WARN  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/langid/lib
   [junit4]   2> 1361405 WARN  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 1361405 WARN  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/velocity/lib
   [junit4]   2> 1361405 WARN  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 1361409 INFO  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 1361409 INFO  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 1361409 WARN  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib
   [junit4]   2> 1361409 INFO  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 1361409 WARN  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 1361409 INFO  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 1361410 WARN  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib
   [junit4]   2> 1361410 WARN  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 1361410 WARN  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib
   [junit4]   2> 1361410 WARN  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/extraction/lib
   [junit4]   2> 1361410 WARN  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 1361410 WARN  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
   [junit4]   2> 1361410 INFO  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1361410 WARN  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/velocity/lib
   [junit4]   2> 1361410 WARN  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/clustering/lib
   [junit4]   2> 1361410 WARN  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 1361411 WARN  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
   [junit4]   2> 1361411 WARN  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/langid/lib
   [junit4]   2> 1361411 WARN  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
   [junit4]   2> 1361411 WARN  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/velocity/lib
   [junit4]   2> 1361411 WARN  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_36F490D22F31B59F-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
   [junit4]   2> 1361416 INFO  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1361418 INFO  (qtp1965882153-12790) [n:127.0.0.1:41065_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1361430 INFO  (qtp1529629596-12795) [n:127.0.0.1:35665_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.s.IndexSchema [AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] Schema name=default-config
   [junit4]   2> 1361452 INFO  (qtp1762186773-12780) [n:127.0.0.1:39453_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.s.IndexSchema [AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] Schema name=default-config
   [junit4]   2> 1361454 INFO  (

[...truncated too long message...]

 Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:34337_solr
   [junit4]   2> 2552229 INFO  (jetty-closer-6632-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 2552229 INFO  (zkCallback-6649-thread-1-processing-n:127.0.0.1:34337_solr) [n:127.0.0.1:34337_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 2552229 INFO  (zkCallback-6647-thread-1-processing-n:127.0.0.1:44761_solr) [n:127.0.0.1:44761_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 2552229 INFO  (jetty-closer-6632-thread-2) [    ] o.a.s.c.Overseer Overseer (id=99397612939509766-127.0.0.1:34337_solr-n_0000000000) closing
   [junit4]   2> 2552229 INFO  (OverseerStateUpdate-99397612939509766-127.0.0.1:34337_solr-n_0000000000) [n:127.0.0.1:34337_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:34337_solr
   [junit4]   2> 2552230 WARN  (OverseerAutoScalingTriggerThread-99397612939509766-127.0.0.1:34337_solr-n_0000000000) [n:127.0.0.1:34337_solr    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 2552231 INFO  (jetty-closer-6632-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@621bf1c4{/solr,null,UNAVAILABLE}
   [junit4]   2> 2552231 INFO  (zkCallback-6647-thread-1-processing-n:127.0.0.1:44761_solr) [n:127.0.0.1:44761_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44761_solr
   [junit4]   2> 2552231 INFO  (jetty-closer-6632-thread-2) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 2552234 INFO  (jetty-closer-6632-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 2552234 INFO  (jetty-closer-6632-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@75dd2cdd: rootName = solr_44761, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@38047dbd
   [junit4]   2> 2552235 INFO  (jetty-closer-6632-thread-1) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:44761_solr
   [junit4]   2> 2552235 INFO  (jetty-closer-6632-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 2552236 INFO  (zkCallback-6647-thread-1-processing-n:127.0.0.1:44761_solr) [n:127.0.0.1:44761_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 2552236 INFO  (jetty-closer-6632-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2552237 INFO  (jetty-closer-6632-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@111ef513{/solr,null,UNAVAILABLE}
   [junit4]   2> 2552237 INFO  (jetty-closer-6632-thread-1) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 2552237 ERROR (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2552238 INFO  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[36F490D22F31B59F]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44065 44065
   [junit4]   2> 2552826 INFO  (Thread-8551) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44065 44065
   [junit4]   2> 2552827 WARN  (Thread-8551) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2>        3       /solr/aliases.json
   [junit4]   2>        2       /solr/security.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2>        3       /solr/clusterstate.json
   [junit4]   2>        3       /solr/collections/timeConfig/state.json
   [junit4]   2>        3       /solr/clusterprops.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2>        3       /solr/live_nodes
   [junit4]   2>        3       /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_36F490D22F31B59F-001
   [junit4]   2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=true): {}, locale=saq, timezone=Asia/Saigon
   [junit4]   2> NOTE: Linux 4.13.0-26-generic amd64/Oracle Corporation 10-ea (64-bit)/cpus=8,threads=1,free=161175608,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [TestPolicyCloud, CleanupOldIndexTest, TestFaceting, MinimalSchemaTest, TestSchemalessBufferedUpdates, TestBlendedInfixSuggestions, TestFiltering, ReturnFieldsTest, SolrCmdDistributorTest, AutoAddReplicasPlanActionTest, CdcrVersionReplicationTest, RegexBoostProcessorTest, DistributedFacetPivotWhiteBoxTest, TestSolrCoreSnapshots, ClassificationUpdateProcessorFactoryTest, TestReplicaProperties, ShardSplitTest, TestTrackingShardHandlerFactory, TestXmlQParserPlugin, DistribJoinFromCollectionTest, TestScoreJoinQPScore, TestFastWriter, TestSSLRandomization, TestCryptoKeys, DistributedSuggestComponentTest, ManagedSchemaRoundRobinCloudTest, AnalysisAfterCoreReloadTest, DisMaxRequestHandlerTest, SolrMetricReporterTest, WordBreakSolrSpellCheckerTest, TestMacros, LeaderElectionContextKeyTest, TestHighFrequencyDictionaryFactory, SolrGraphiteReporterTest, TestConfigSetsAPI, TestSolrCoreProperties, TemplateUpdateProcessorTest, TestMiniSolrCloudClusterSSL, CoreAdminCreateDiscoverTest, TestGraphTermsQParserPlugin, CdcrReplicationDistributedZkTest, ReplicationFactorTest, TestSolrDeletionPolicy2, TestClusterStateMutator, TestPushWriter, TestAnalyzedSuggestions, TestQuerySenderListener, RecoveryAfterSoftCommitTest, URLClassifyProcessorTest, TestInfoStreamLogging, InfoHandlerTest, TestGeoJSONResponseWriter, TestMultiValuedNumericRangeQuery, TestPKIAuthenticationPlugin, TestJsonFacets, TestBulkSchemaAPI, ReplicaListTransformerTest, TestReloadAndDeleteDocs, TestValueSourceCache, TestLazyCores, ConfigureRecoveryStrategyTest, ClusterStateTest, BooleanFieldTest, AnalyticsMergeStrategyTest, TestRestoreCore, RecoveryZkTest, StatelessScriptUpdateProcessorFactoryTest, OverseerCollectionConfigSetProcessorTest, BasicDistributedZk2Test, TestDownShardTolerantSearch, TestStressReorder, PingRequestHandlerTest, TestInPlaceUpdatesStandalone, CreateCollectionCleanupTest, TestDistribStateManager, TriggerIntegrationTest, SimpleMLTQParserTest, SolrCLIZkUtilsTest, NodeMutatorTest, NoCacheHeaderTest, TestLFUCache, DeleteStatusTest, IndexSchemaRuntimeFieldTest, TestLeaderElectionZkExpiry, TestSolrConfigHandler, AliasIntegrationTest, LeaderInitiatedRecoveryOnCommitTest, ClusterStateUpdateTest, TestDocTermOrds, MetricTriggerTest, TestMultiWordSynonyms, WrapperMergePolicyFactoryTest, TestAtomicUpdateErrorCases, ExitableDirectoryReaderTest, TestDelegationWithHadoopAuth, HdfsUnloadDistributedZkTest, TestStressInPlaceUpdates, ImplicitSnitchTest, MoveReplicaHDFSTest, TestRandomCollapseQParserPlugin, TestCodecSupport, DistributedSpellCheckComponentTest, MultiTermTest, TestBlobHandler, TimeZoneUtilsTest, CopyFieldTest, TestNumericTerms64, V2ApiIntegrationTest, TestCloudJSONFacetJoinDomain, ExecutePlanActionTest, TestExpandComponent, CollectionTooManyReplicasTest, SpellCheckComponentTest, SearchHandlerTest, DocValuesNotIndexedTest, TestUtils, TestBadConfig, DirectSolrSpellCheckerTest, TestSortByMinMaxFunction, DistanceFunctionTest, SparseHLLTest, TestPayloadCheckQParserPlugin, TestUpdate, TestDFRSimilarityFactory, BaseCdcrDistributedZkTest, MoreLikeThisHandlerTest, TestQueryTypes, ChaosMonkeyNothingIsSafeWithPullReplicasTest, LeaderInitiatedRecoveryOnShardRestartTest, MigrateRouteKeyTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerModifyCollectionTest, PeerSyncReplicationTest, TestCloudSearcherWarming, TestRebalanceLeaders, TestSizeLimitedDistributedMap, TestSkipOverseerOperations, TestTolerantUpdateProcessorCloud, VMParamsZkACLAndCredentialsProvidersTest, CollectionsAPIAsyncDistributedZkTest, TestLocalFSCloudBackupRestore, NodeAddedTriggerTest, SystemLogListenerTest, TestComputePlanAction, TestLargeCluster, HdfsRecoveryZkTest, HdfsTlogReplayBufferedWhileIndexingTest, StressHdfsTest, TestInitParams, TestShardHandlerFactory, TestSolrCloudSnapshots, TestSQLHandler, PropertiesRequestHandlerTest, SegmentsInfoRequestHandlerTest, TestConfigsApi, CustomHighlightComponentTest, DistributedQueryComponentOptimizationTest, SolrMetricsIntegrationTest, SolrJmxReporterTest, SolrSlf4jReporterTest, SolrShardReporterTest, RegexBytesRefFilterTest, SubstringBytesRefFilterTest, TestIntervalFaceting, SpatialRPTFieldTypeTest, TestCloudManagedSchema, CurrencyRangeFacetCloudTest, TestComplexPhraseQParserPlugin, TestCustomSort, TestFieldSortValues, TestInitQParser, TestMissingGroups, TestSearcherReuse, TestSimpleQParserPlugin, TestSmileRequest, TestSolrCoreParser, TestStressUserVersions, TestTrieFacet, TestJsonFacetRefinement, TestMinMaxOnMultiValuedField, GraphQueryTest, TestNestedDocsSort, TestJsonRequest, CloudMLTQParserTest, TestDFISimilarityFactory, TestDefaultStatsCache, TestExactStatsCache, TestZkAclsWithHadoopAuth, ConjunctionSolrSpellCheckerTest, SpellCheckCollatorWithCollapseTest, BlockDirectoryTest, BufferStoreTest, TestFieldCache, TestFieldCacheSort, TestFieldCacheWithThreads, TestUninvertingReader, AnalysisErrorHandlingTest, PeerSyncWithIndexFingerprintCachingTest, AtomicUpdateProcessorFactoryTest, CloneFieldUpdateProcessorFactoryTest, PreAnalyzedUpdateProcessorTest, TimeRoutedAliasUpdateProcessorTest]
   [junit4] Completed [760/767 (2!)] on J2 in 33.09s, 2 tests, 1 error <<< FAILURES!

[...truncated 40910 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk1.8.0_144) - Build # 1227 - Failure!

Policeman Jenkins Server-2
In reply to this post by Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1227/
Java: 64bit/jdk1.8.0_144 -XX:+UseCompressedOops -XX:+UseSerialGC

All tests passed

Build Log:
[...truncated 37448 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:836: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:117: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build.xml:388: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/common-build.xml:2039: Rat problems were found!

Total time: 84 minutes 57 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-10-ea+37) - Build # 1228 - Still unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1228/
Java: 64bit/jdk-10-ea+37 -XX:-UseCompressedOops -XX:+UseG1GC

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicZkTest

Error Message:
SolrCore.getOpenCount()==2

Stack Trace:
java.lang.RuntimeException: SolrCore.getOpenCount()==2
        at __randomizedtesting.SeedInfo.seed([598DE416FDC2ED70]:0)
        at org.apache.solr.util.TestHarness.close(TestHarness.java:379)
        at org.apache.solr.SolrTestCaseJ4.deleteCore(SolrTestCaseJ4.java:792)
        at org.apache.solr.cloud.AbstractZkTestCase.azt_afterClass(AbstractZkTestCase.java:147)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:897)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicZkTest

Error Message:
SolrCore.getOpenCount()==2

Stack Trace:
java.lang.RuntimeException: SolrCore.getOpenCount()==2
        at __randomizedtesting.SeedInfo.seed([598DE416FDC2ED70]:0)
        at org.apache.solr.util.TestHarness.close(TestHarness.java:379)
        at org.apache.solr.SolrTestCaseJ4.deleteCore(SolrTestCaseJ4.java:792)
        at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:288)
        at jdk.internal.reflect.GeneratedMethodAccessor59.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:897)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicZkTest

Error Message:
27 threads leaked from SUITE scope at org.apache.solr.cloud.BasicZkTest:     1) Thread[id=10422, name=SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker-SendThread(127.0.0.1:37597), state=RUNNABLE, group=TGRP-BasicZkTest]         at java.base@10-ea/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at java.base@10-ea/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265)         at java.base@10-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)         at java.base@10-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:89)         at java.base@10-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:100)         at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)    2) Thread[id=10433, name=Thread-2605, state=WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/java.lang.Object.wait(Native Method)         at java.base@10-ea/java.lang.Object.wait(Object.java:328)         at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:1722)    3) Thread[id=10423, name=SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker-EventThread, state=WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)         at java.base@10-ea/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)         at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)    4) Thread[id=10428, name=OverseerAutoScalingTriggerThread-99400183216013315-88.99.242.108:8983_-n_0000000000, state=WAITING, group=Overseer autoscaling triggers]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)         at app//org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:166)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)    5) Thread[id=10443, name=Thread-2606, state=WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/java.lang.Object.wait(Native Method)         at java.base@10-ea/java.lang.Thread.join(Thread.java:1353)         at java.base@10-ea/java.lang.Thread.join(Thread.java:1427)         at app//org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:297)         at app//org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:310)         at app//org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:491)    6) Thread[id=10448, name=zkCallback-2244-thread-5, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)    7) Thread[id=10438, name=coreZkRegister-4354-thread-1, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)    8) Thread[id=10429, name=zkCallback-2244-thread-1, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)    9) Thread[id=10449, name=zkCallback-2244-thread-6, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   10) Thread[id=10424, name=zkConnectionManagerCallback-2245-thread-1, state=WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)         at java.base@10-ea/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   11) Thread[id=10425, name=OverseerStateUpdate-99400183216013315-88.99.242.108:8983_-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2117)         at app//org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:463)         at app//org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:205)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   12) Thread[id=10426, name=Connection evictor, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   13) Thread[id=10444, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:37597, state=RUNNABLE, group=TGRP-BasicZkTest]         at java.base@10-ea/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at java.base@10-ea/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265)         at java.base@10-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)         at java.base@10-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:89)         at java.base@10-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:100)         at app//org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:173)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   14) Thread[id=10431, name=ScheduledTrigger-4359-thread-2, state=WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   15) Thread[id=10440, name=zkCallback-2244-thread-3, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   16) Thread[id=10442, name=ScheduledTrigger-4359-thread-4, state=WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   17) Thread[id=10418, name=Connection evictor, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   18) Thread[id=10430, name=ScheduledTrigger-4359-thread-1, state=WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   19) Thread[id=10434, name=watches-2246-thread-1, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   20) Thread[id=10427, name=OverseerCollectionConfigSetProcessor-99400183216013315-88.99.242.108:8983_-n_0000000000, state=TIMED_WAITING, group=Overseer collection creation process.]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2117)         at app//org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:463)         at app//org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:235)         at app//org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:228)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   21) Thread[id=10439, name=zkCallback-2244-thread-2, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   22) Thread[id=10445, name=SessionTracker, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/java.lang.Object.wait(Native Method)         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)   23) Thread[id=10437, name=searcherExecutor-4364-thread-1, state=WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)         at java.base@10-ea/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   24) Thread[id=10447, name=ProcessThread(sid:0 cport:37597):, state=WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)         at java.base@10-ea/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)         at app//org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:122)   25) Thread[id=10417, name=Connection evictor, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   26) Thread[id=10435, name=ScheduledTrigger-4359-thread-3, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2117)         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   27) Thread[id=10441, name=zkCallback-2244-thread-4, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 27 threads leaked from SUITE scope at org.apache.solr.cloud.BasicZkTest:
   1) Thread[id=10422, name=SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker-SendThread(127.0.0.1:37597), state=RUNNABLE, group=TGRP-BasicZkTest]
        at java.base@10-ea/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at java.base@10-ea/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265)
        at java.base@10-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)
        at java.base@10-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:89)
        at java.base@10-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:100)
        at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
        at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
   2) Thread[id=10433, name=Thread-2605, state=WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/java.lang.Object.wait(Native Method)
        at java.base@10-ea/java.lang.Object.wait(Object.java:328)
        at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:1722)
   3) Thread[id=10423, name=SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker-EventThread, state=WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
        at java.base@10-ea/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
        at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
   4) Thread[id=10428, name=OverseerAutoScalingTriggerThread-99400183216013315-88.99.242.108:8983_-n_0000000000, state=WAITING, group=Overseer autoscaling triggers]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
        at app//org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:166)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   5) Thread[id=10443, name=Thread-2606, state=WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/java.lang.Object.wait(Native Method)
        at java.base@10-ea/java.lang.Thread.join(Thread.java:1353)
        at java.base@10-ea/java.lang.Thread.join(Thread.java:1427)
        at app//org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:297)
        at app//org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:310)
        at app//org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:491)
   6) Thread[id=10448, name=zkCallback-2244-thread-5, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   7) Thread[id=10438, name=coreZkRegister-4354-thread-1, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   8) Thread[id=10429, name=zkCallback-2244-thread-1, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   9) Thread[id=10449, name=zkCallback-2244-thread-6, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  10) Thread[id=10424, name=zkConnectionManagerCallback-2245-thread-1, state=WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
        at java.base@10-ea/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  11) Thread[id=10425, name=OverseerStateUpdate-99400183216013315-88.99.242.108:8983_-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2117)
        at app//org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:463)
        at app//org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:205)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  12) Thread[id=10426, name=Connection evictor, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/java.lang.Thread.sleep(Native Method)
        at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  13) Thread[id=10444, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:37597, state=RUNNABLE, group=TGRP-BasicZkTest]
        at java.base@10-ea/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at java.base@10-ea/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265)
        at java.base@10-ea/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)
        at java.base@10-ea/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:89)
        at java.base@10-ea/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:100)
        at app//org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:173)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  14) Thread[id=10431, name=ScheduledTrigger-4359-thread-2, state=WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
        at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
        at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  15) Thread[id=10440, name=zkCallback-2244-thread-3, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  16) Thread[id=10442, name=ScheduledTrigger-4359-thread-4, state=WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
        at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
        at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  17) Thread[id=10418, name=Connection evictor, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/java.lang.Thread.sleep(Native Method)
        at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  18) Thread[id=10430, name=ScheduledTrigger-4359-thread-1, state=WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
        at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
        at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  19) Thread[id=10434, name=watches-2246-thread-1, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  20) Thread[id=10427, name=OverseerCollectionConfigSetProcessor-99400183216013315-88.99.242.108:8983_-n_0000000000, state=TIMED_WAITING, group=Overseer collection creation process.]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2117)
        at app//org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:463)
        at app//org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:235)
        at app//org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:228)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  21) Thread[id=10439, name=zkCallback-2244-thread-2, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  22) Thread[id=10445, name=SessionTracker, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/java.lang.Object.wait(Native Method)
        at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
  23) Thread[id=10437, name=searcherExecutor-4364-thread-1, state=WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
        at java.base@10-ea/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  24) Thread[id=10447, name=ProcessThread(sid:0 cport:37597):, state=WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
        at java.base@10-ea/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
        at app//org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:122)
  25) Thread[id=10417, name=Connection evictor, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/java.lang.Thread.sleep(Native Method)
        at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  26) Thread[id=10435, name=ScheduledTrigger-4359-thread-3, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2117)
        at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  27) Thread[id=10441, name=zkCallback-2244-thread-4, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
        at __randomizedtesting.SeedInfo.seed([598DE416FDC2ED70]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=10422, name=SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker-SendThread(127.0.0.1:37597), state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/java.lang.Thread.sleep(Native Method)         at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)    2) Thread[id=10433, name=Thread-2605, state=WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/java.lang.Object.wait(Native Method)         at java.base@10-ea/java.lang.Object.wait(Object.java:328)         at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:1722)    3) Thread[id=10430, name=ScheduledTrigger-4359-thread-1, state=WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)    4) Thread[id=10434, name=watches-2246-thread-1, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)    5) Thread[id=10439, name=zkCallback-2244-thread-2, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)    6) Thread[id=10448, name=zkCallback-2244-thread-5, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)    7) Thread[id=10438, name=coreZkRegister-4354-thread-1, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)    8) Thread[id=10437, name=searcherExecutor-4364-thread-1, state=WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)         at java.base@10-ea/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)    9) Thread[id=10429, name=zkCallback-2244-thread-1, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   10) Thread[id=10449, name=zkCallback-2244-thread-6, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   11) Thread[id=10424, name=zkConnectionManagerCallback-2245-thread-1, state=WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)         at java.base@10-ea/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   12) Thread[id=10431, name=ScheduledTrigger-4359-thread-2, state=WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   13) Thread[id=10440, name=zkCallback-2244-thread-3, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   14) Thread[id=10435, name=ScheduledTrigger-4359-thread-3, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2117)         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   15) Thread[id=10441, name=zkCallback-2244-thread-4, state=TIMED_WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)   16) Thread[id=10442, name=ScheduledTrigger-4359-thread-4, state=WAITING, group=TGRP-BasicZkTest]         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=10422, name=SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker-SendThread(127.0.0.1:37597), state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/java.lang.Thread.sleep(Native Method)
        at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)
   2) Thread[id=10433, name=Thread-2605, state=WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/java.lang.Object.wait(Native Method)
        at java.base@10-ea/java.lang.Object.wait(Object.java:328)
        at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:1722)
   3) Thread[id=10430, name=ScheduledTrigger-4359-thread-1, state=WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
        at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
        at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   4) Thread[id=10434, name=watches-2246-thread-1, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   5) Thread[id=10439, name=zkCallback-2244-thread-2, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   6) Thread[id=10448, name=zkCallback-2244-thread-5, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   7) Thread[id=10438, name=coreZkRegister-4354-thread-1, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   8) Thread[id=10437, name=searcherExecutor-4364-thread-1, state=WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
        at java.base@10-ea/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   9) Thread[id=10429, name=zkCallback-2244-thread-1, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  10) Thread[id=10449, name=zkCallback-2244-thread-6, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  11) Thread[id=10424, name=zkConnectionManagerCallback-2245-thread-1, state=WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
        at java.base@10-ea/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  12) Thread[id=10431, name=ScheduledTrigger-4359-thread-2, state=WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
        at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
        at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  13) Thread[id=10440, name=zkCallback-2244-thread-3, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  14) Thread[id=10435, name=ScheduledTrigger-4359-thread-3, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2117)
        at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  15) Thread[id=10441, name=zkCallback-2244-thread-4, state=TIMED_WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
  16) Thread[id=10442, name=ScheduledTrigger-4359-thread-4, state=WAITING, group=TGRP-BasicZkTest]
        at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
        at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
        at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
        at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
        at __randomizedtesting.SeedInfo.seed([598DE416FDC2ED70]:0)




Build Log:
[...truncated 12832 lines...]
   [junit4] Suite: org.apache.solr.cloud.BasicZkTest
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.BasicZkTest_598DE416FDC2ED70-001/init-core-data-001
   [junit4]   2> 1141590 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1141591 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
   [junit4]   2> 1141591 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1141591 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1141592 INFO  (Thread-2604) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1141592 INFO  (Thread-2604) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1141592 ERROR (Thread-2604) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1141692 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:37597
   [junit4]   2> 1141693 INFO  (zkConnectionManagerCallback-2237-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1141694 INFO  (zkConnectionManagerCallback-2239-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1141696 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1141696 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1141697 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1141697 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1141698 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1141698 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1141698 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1141699 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1141699 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1141700 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1141700 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1141701 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 1141701 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1141713 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1141724 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 1141772 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1141780 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37597/solr
   [junit4]   2> 1141781 INFO  (zkConnectionManagerCallback-2243-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1141785 INFO  (zkConnectionManagerCallback-2245-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1141809 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [n:88.99.242.108:8983_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1141810 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [n:88.99.242.108:8983_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 88.99.242.108:8983_
   [junit4]   2> 1141810 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [n:88.99.242.108:8983_    ] o.a.s.c.Overseer Overseer (id=99400183216013315-88.99.242.108:8983_-n_0000000000) starting
   [junit4]   2> 1141812 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [n:88.99.242.108:8983_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/88.99.242.108:8983_
   [junit4]   2> 1141813 INFO  (zkCallback-2244-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1141835 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [n:88.99.242.108:8983_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4b510857
   [junit4]   2> 1141839 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [n:88.99.242.108:8983_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4b510857
   [junit4]   2> 1141839 INFO  (SUITE-BasicZkTest-seed#[598DE416FDC2ED70]-worker) [n:88.99.242.108:8983_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4b510857
   [junit4]   2> 1141841 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1142843 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c:collection1 s:shard1  x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1142878 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c:collection1 s:shard1  x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1142899 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c:collection1 s:shard1  x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1143019 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c:collection1 s:shard1  x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1143027 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c:collection1 s:shard1  x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
   [junit4]   2> 1143027 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c:collection1 s:shard1  x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.88.99.242.108:8983__collection1' (registry 'solr.core.collection1.shard1.88.99.242.108:8983__collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4b510857
   [junit4]   2> 1143027 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c:collection1 s:shard1  x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1143027 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c:collection1 s:shard1  x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.BasicZkTest_598DE416FDC2ED70-001/init-core-data-001/]
   [junit4]   2> 1143028 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c:collection1 s:shard1  x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=27, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.10104465363566827]
   [junit4]   2> 1143054 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c:collection1 s:shard1  x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1143054 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c:collection1 s:shard1  x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1143055 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c:collection1 s:shard1  x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1143055 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c:collection1 s:shard1  x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1143056 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c:collection1 s:shard1  x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=84.052734375, floorSegmentMB=1.21484375, forceMergeDeletesPctAllowed=6.562405319260285, segmentsPerTier=38.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1143056 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c:collection1 s:shard1  x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3d8e42c6[collection1] main]
   [junit4]   2> 1143057 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c:collection1 s:shard1  x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1143192 INFO  (coreLoadExecutor-4363-thread-1-processing-n:88.99.242.108:8983_) [n:88.99.242.108:8983_ c

[...truncated too long message...]

java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
   [junit4]    >         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   [junit4]    >    9) Thread[id=10429, name=zkCallback-2244-thread-1, state=TIMED_WAITING, group=TGRP-BasicZkTest]
   [junit4]    >         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
   [junit4]    >         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   [junit4]    >   10) Thread[id=10449, name=zkCallback-2244-thread-6, state=TIMED_WAITING, group=TGRP-BasicZkTest]
   [junit4]    >         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
   [junit4]    >         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   [junit4]    >   11) Thread[id=10424, name=zkConnectionManagerCallback-2245-thread-1, state=WAITING, group=TGRP-BasicZkTest]
   [junit4]    >         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
   [junit4]    >         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   [junit4]    >   12) Thread[id=10431, name=ScheduledTrigger-4359-thread-2, state=WAITING, group=TGRP-BasicZkTest]
   [junit4]    >         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
   [junit4]    >         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   [junit4]    >   13) Thread[id=10440, name=zkCallback-2244-thread-3, state=TIMED_WAITING, group=TGRP-BasicZkTest]
   [junit4]    >         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
   [junit4]    >         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   [junit4]    >   14) Thread[id=10435, name=ScheduledTrigger-4359-thread-3, state=TIMED_WAITING, group=TGRP-BasicZkTest]
   [junit4]    >         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2117)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
   [junit4]    >         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   [junit4]    >   15) Thread[id=10441, name=zkCallback-2244-thread-4, state=TIMED_WAITING, group=TGRP-BasicZkTest]
   [junit4]    >         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
   [junit4]    >         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   [junit4]    >   16) Thread[id=10442, name=ScheduledTrigger-4359-thread-4, state=WAITING, group=TGRP-BasicZkTest]
   [junit4]    >         at java.base@10-ea/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
   [junit4]    >         at java.base@10-ea/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
   [junit4]    >         at java.base@10-ea/java.lang.Thread.run(Thread.java:844)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([598DE416FDC2ED70]:0)
   [junit4] Completed [345/767 (1!)] on J1 in 31.57s, 1 test, 4 errors <<< FAILURES!

[...truncated 42278 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-10-ea+37) - Build # 1229 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1229/
Java: 64bit/jdk-10-ea+37 -XX:+UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testFuzzyKmeans

Error Message:
expected:<1.3724537218127956E-4> but was:<0.0>

Stack Trace:
java.lang.AssertionError: expected:<1.3724537218127956E-4> but was:<0.0>
        at __randomizedtesting.SeedInfo.seed([6471768B25648F68:FEA595A4E052219]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:443)
        at org.junit.Assert.assertEquals(Assert.java:512)
        at org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testFuzzyKmeans(StreamExpressionTest.java:7140)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost

Error Message:


Stack Trace:
java.util.concurrent.TimeoutException
        at __randomizedtesting.SeedInfo.seed([8A7B7719C587386D:356EB9E7466D5DEB]:0)
        at org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.waitForState(SimSolrCloudTestCase.java:271)
        at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.doTestNodeLost(TestLargeCluster.java:453)
        at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testNodeLost(TestLargeCluster.java:295)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 13657 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
   [junit4]   2> 2151391 INFO  (SUITE-TestLargeCluster-seed#[8A7B7719C587386D]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestLargeCluster_8A7B7719C587386D-001/init-core-data-001
   [junit4]   2> 2151392 INFO  (SUITE-TestLargeCluster-seed#[8A7B7719C587386D]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 2151392 INFO  (SUITE-TestLargeCluster-seed#[8A7B7719C587386D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
   [junit4]   2> 2151393 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 2151393 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 2151394 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10022_solr, 127.0.0.1:10018_solr, 127.0.0.1:10016_solr, 127.0.0.1:10021_solr, 127.0.0.1:10017_solr, 127.0.0.1:10020_solr, 127.0.0.1:10019_solr, 127.0.0.1:10023_solr]
   [junit4]   2> 2151394 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 2151394 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 2151394 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2151394 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 2151394 DEBUG (ScheduledTrigger-8416-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 16
   [junit4]   2> 2151400 INFO  (TEST-TestLargeCluster.testBasic-seed#[8A7B7719C587386D]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testBasic
   [junit4]   2> 2151400 DEBUG (TEST-TestLargeCluster.testBasic-seed#[8A7B7719C587386D]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 2151400 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 2151400 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 2151400 INFO  (TEST-TestLargeCluster.testBasic-seed#[8A7B7719C587386D]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 2151400 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2151401 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException
   [junit4]   2> at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1259)
   [junit4]   2> at java.base/java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:317)
   [junit4]   2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:154)
   [junit4]   2> at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 2151401 DEBUG (TEST-TestLargeCluster.testBasic-seed#[8A7B7719C587386D]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 2151401 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 2151401 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 2151401 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10104_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10105_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10102_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10101_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10107_solr, 127.0.0.1:10055_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10108_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10114_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10111_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10112_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10113_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10030_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10115_solr, 127.0.0.1:10069_solr, 127.0.0.1:10110_solr, 127.0.0.1:10062_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10106_solr, 127.0.0.1:10100_solr, 127.0.0.1:10037_solr, 127.0.0.1:10109_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10103_solr, 127.0.0.1:10072_solr, 127.0.0.1:10056_solr]
   [junit4]   2> 2151401 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 2151401 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 2151401 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2151401 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 2151402 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151422 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151442 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151463 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151483 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151503 DEBUG (simCloudManagerPool-8418-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 2151503 DEBUG (simCloudManagerPool-8418-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 2151503 DEBUG (simCloudManagerPool-8418-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10104_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10105_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10102_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10101_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10107_solr, 127.0.0.1:10055_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10108_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10114_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10111_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10112_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10113_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10030_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10115_solr, 127.0.0.1:10069_solr, 127.0.0.1:10110_solr, 127.0.0.1:10062_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10106_solr, 127.0.0.1:10100_solr, 127.0.0.1:10037_solr, 127.0.0.1:10109_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10103_solr, 127.0.0.1:10072_solr, 127.0.0.1:10056_solr]
   [junit4]   2> 2151503 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151503 DEBUG (simCloudManagerPool-8418-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10104_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10105_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10102_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10101_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10107_solr, 127.0.0.1:10055_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10108_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10114_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10111_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10112_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10113_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10030_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10115_solr, 127.0.0.1:10069_solr, 127.0.0.1:10110_solr, 127.0.0.1:10062_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10106_solr, 127.0.0.1:10100_solr, 127.0.0.1:10037_solr, 127.0.0.1:10109_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10103_solr, 127.0.0.1:10072_solr, 127.0.0.1:10056_solr]
   [junit4]   2> 2151503 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 2151504 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2151504 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151504 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 2151505 DEBUG (simCloudManagerPool-8418-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 2151505 DEBUG (simCloudManagerPool-8418-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 2151505 DEBUG (simCloudManagerPool-8418-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10104_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10105_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10102_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10101_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10107_solr, 127.0.0.1:10055_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10108_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10114_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10111_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10112_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10113_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10030_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10115_solr, 127.0.0.1:10069_solr, 127.0.0.1:10110_solr, 127.0.0.1:10062_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10106_solr, 127.0.0.1:10100_solr, 127.0.0.1:10037_solr, 127.0.0.1:10109_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10103_solr, 127.0.0.1:10072_solr, 127.0.0.1:10056_solr]
   [junit4]   2> 2151505 DEBUG (simCloudManagerPool-8418-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10104_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10105_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10102_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10101_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10107_solr, 127.0.0.1:10055_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10108_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10114_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10111_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10112_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10113_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10030_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10115_solr, 127.0.0.1:10069_solr, 127.0.0.1:10110_solr, 127.0.0.1:10062_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10106_solr, 127.0.0.1:10100_solr, 127.0.0.1:10037_solr, 127.0.0.1:10109_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10103_solr, 127.0.0.1:10072_solr, 127.0.0.1:10056_solr]
   [junit4]   2> 2151506 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 2151506 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2151506 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 2151524 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151524 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151544 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151544 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151564 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151564 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151585 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151585 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151605 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151605 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151625 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151625 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151645 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151645 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151666 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151666 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151686 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151686 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151706 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151706 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151726 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151726 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151747 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151747 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151767 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151767 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151787 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151787 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151807 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151807 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151828 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151828 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151848 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151848 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151868 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151868 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151889 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151889 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151909 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151909 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151929 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151929 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151949 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151949 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151970 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151970 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2151990 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2151990 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152010 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152010 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152031 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152031 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152051 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152051 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152071 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152071 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152091 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152091 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152112 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152112 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152132 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152132 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152152 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152152 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152172 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152172 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152193 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152193 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152213 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152213 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152233 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152233 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152253 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152253 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152274 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152274 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152294 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152294 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152314 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152314 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152335 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152335 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152355 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152355 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152375 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 2152375 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
   [junit4]   2> 2152377 INFO  (TEST-TestLargeCluster.testBasic-seed#[8A7B7719C587386D]) [    ] o.a.s.c.a.s.TestLargeCluster Ready after 37704ms
   [junit4]   2> 2152381 DEBUG (simCloudManagerPool-8418-thread-79) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard2: {"core_node18":{
   [junit4]   2>     "node_name":"127.0.0.1:10023_solr",
   [junit4]   2>     "core":"testBasic_shard2_replica_n18",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInBytes":123450000,
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testBasic"}}
   [junit4]   2> 2152381 DEBUG (simCloudManagerPool-8418-thread-80) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard3: {"core_node39":{
   [junit4]   2>     "node_name":"127.0.0.1:10104_solr",
   [junit4]   2>     "core":"testBasic_shard3_replica_t39",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"TLOG",
   [junit4]   2>     "INDEX.sizeInBytes":123450000,
   [junit4]   2>     "shard":"shard3",
   [junit4]   2>     "collection":"testBasic"}}
   [junit4]   2> 2152381 DEBUG (simCloudManagerPool-8418-thread-83) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard1: {"core_node1":{
   [junit4]   2>     "node_name":"127.0.0.1:10101_solr",
   [junit4]   2>     "core":"testBasic_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInBytes":123450000,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testBasic"}}
   [junit4]   2> 2152381 DEBUG (simCloudManagerPool-8418-thread-81) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard4: {"core_node54":{
   [junit4]   2>     "node_name":"127.0.0.1:10064_solr",
   [junit4]   2>     "core":"testBasic_shard4_replica_t54",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"TLOG",
   [junit4]   2>     "INDEX.sizeInBytes":123450000,
   [junit4]   2>     "shard":"shard4",
   [junit4]   2>     "collection":"testBasic"}}
   [junit4]   2> 2152382 DEBUG (simCloudManagerPool-8418-thread-82) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard5: {"core_node65":{
   [junit4]   2>     "node_name":"127.0.0.1:10035_solr",
   [junit4]   2>     "core":"testBasic_shard5_replica_n65",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInBytes":123450000,
   [junit4]   2>     "shard":"shard5",
   [junit4]   2>     "collection":"testBasic"}}
   [junit4]   2> 2152395 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 96
   [junit4]   2> 2152395 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 96
   [junit4]   2> 2152396 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10019_solr
   [junit4]   2> 2152396 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10019_solr
   [junit4]   2> 2152396 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10105_solr
   [junit4]   2> 2152396 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10105_solr
   [junit4]   2> 2152396 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10052_solr
   [junit4]   2> 2152396 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10026_solr
   [junit4]   2> 2152396 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10052_solr
   [junit4]   2> 2152396 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10026_solr
   [junit4]   2> 2152416 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 92
   [junit4]   2> 2152416 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10022_solr
   [junit4]   2> 2152416 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10036_solr
   [junit4]   2> 2152416 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10071_solr
   [junit4]   2> 2152416 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10076_solr
   [junit4]   2> 2152416 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 92
   [junit4]   2> 2152416 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10022_solr
   [junit4]   2> 2152416 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10036_solr
   [junit4]   2> 2152416 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10071_solr
   [junit4]   2> 2152416 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10076_solr
   [junit4]   2> 2152436 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 92
   [junit4]   2> 2152437 DEBUG (ScheduledTrigger-8419-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 92
   [junit4]   2> 2152457 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 92
   [junit4]   2> 2152457 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 92
   [junit4]   2> 2152477 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 92
   [junit4]   2> 2152477 DEBUG (ScheduledTrigger-8419-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 92
   [junit4]   2> 2152497 DEBUG (ScheduledTrigger-8419-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 92
   [junit4]   2> 2152497 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 92
   [junit4]   2> 2152497 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10052_solr, 127.0.0.1:10105_solr, 127.0.0.1:10026_solr, 127.0.0.1:10019_solr]
   [junit4]   2> 2152497 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"7c25ef310339Tbb8x7tcy164rfweb08jol5m0l",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":136502368600889,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[
   [junit4]   2>       136502368600889,
   [junit4]   2>       136502363258989,
   [junit4]   2>       136502370026839,
   [junit4]   2>       136502359925389],
   [junit4]   2>     "nodeNames":[
   [junit4]   2>       "127.0.0.1:10052_solr",
   [junit4]   2>       "127.0.0.1:10105_solr",
   [junit4]   2>       "127.0.0.1:10026_solr",
   [junit4]   2>       "127.0.0.1:10019_solr"]}}
   [junit4]   2> 2152498 DEBUG (ScheduledTrigger-8419-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger]
   [junit4]   2> 2152498 DEBUG (AutoscalingActionExecutor-8420-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"7c25ef310339