[JENKINS] Lucene-Solr-Tests-master - Build # 2530 - Unstable

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2530 - Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2530/

2 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration

Error Message:
did not finish processing in time

Stack Trace:
java.lang.AssertionError: did not finish processing in time
        at __randomizedtesting.SeedInfo.seed([633326200D19E9FD:308A6490EF087C07]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration(IndexSizeTriggerTest.java:425)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger

Error Message:
waitFor not elapsed but produced an event

Stack Trace:
java.lang.AssertionError: waitFor not elapsed but produced an event
        at __randomizedtesting.SeedInfo.seed([633326200D19E9FD:F810A294D69AD0]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:180)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12430 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
   [junit4]   2> 217288 INFO  (SUITE-IndexSizeTriggerTest-seed#[633326200D19E9FD]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_633326200D19E9FD-001/init-core-data-001
   [junit4]   2> 217293 WARN  (SUITE-IndexSizeTriggerTest-seed#[633326200D19E9FD]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=3 numCloses=3
   [junit4]   2> 217293 INFO  (SUITE-IndexSizeTriggerTest-seed#[633326200D19E9FD]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 217295 INFO  (SUITE-IndexSizeTriggerTest-seed#[633326200D19E9FD]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 217296 INFO  (SUITE-IndexSizeTriggerTest-seed#[633326200D19E9FD]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_633326200D19E9FD-001/tempDir-001
   [junit4]   2> 217296 INFO  (SUITE-IndexSizeTriggerTest-seed#[633326200D19E9FD]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 217308 INFO  (Thread-164) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 217308 INFO  (Thread-164) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 217367 ERROR (Thread-164) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 217404 INFO  (SUITE-IndexSizeTriggerTest-seed#[633326200D19E9FD]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:53192
   [junit4]   2> 217454 INFO  (zkConnectionManagerCallback-126-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 217493 INFO  (jetty-launcher-123-thread-1) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 217546 INFO  (jetty-launcher-123-thread-2) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 217591 INFO  (jetty-launcher-123-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 217591 INFO  (jetty-launcher-123-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 217591 INFO  (jetty-launcher-123-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 217602 INFO  (jetty-launcher-123-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@40a2d020{/solr,null,AVAILABLE}
   [junit4]   2> 217604 INFO  (jetty-launcher-123-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7f579a1c{SSL,[ssl, http/1.1]}{127.0.0.1:54410}
   [junit4]   2> 217604 INFO  (jetty-launcher-123-thread-1) [    ] o.e.j.s.Server Started @217728ms
   [junit4]   2> 217605 INFO  (jetty-launcher-123-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54410}
   [junit4]   2> 217605 ERROR (jetty-launcher-123-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 217605 INFO  (jetty-launcher-123-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 217605 INFO  (jetty-launcher-123-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 217605 INFO  (jetty-launcher-123-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 217605 INFO  (jetty-launcher-123-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 217605 INFO  (jetty-launcher-123-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-16T16:24:31.024Z
   [junit4]   2> 217643 INFO  (jetty-launcher-123-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 217643 INFO  (jetty-launcher-123-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 217643 INFO  (jetty-launcher-123-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 217652 INFO  (jetty-launcher-123-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@552d7e03{/solr,null,AVAILABLE}
   [junit4]   2> 217654 INFO  (jetty-launcher-123-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@57ca6425{SSL,[ssl, http/1.1]}{127.0.0.1:44548}
   [junit4]   2> 217654 INFO  (jetty-launcher-123-thread-2) [    ] o.e.j.s.Server Started @217777ms
   [junit4]   2> 217654 INFO  (jetty-launcher-123-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44548}
   [junit4]   2> 217654 ERROR (jetty-launcher-123-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 217654 INFO  (jetty-launcher-123-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 217654 INFO  (jetty-launcher-123-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 217654 INFO  (jetty-launcher-123-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 217654 INFO  (jetty-launcher-123-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 217655 INFO  (jetty-launcher-123-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-16T16:24:31.074Z
   [junit4]   2> 217716 INFO  (zkConnectionManagerCallback-130-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 217728 INFO  (jetty-launcher-123-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 217746 INFO  (zkConnectionManagerCallback-128-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 217747 INFO  (jetty-launcher-123-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 217777 INFO  (jetty-launcher-123-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53192/solr
   [junit4]   2> 217809 INFO  (jetty-launcher-123-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53192/solr
   [junit4]   2> 217832 INFO  (zkConnectionManagerCallback-134-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 217857 INFO  (zkConnectionManagerCallback-138-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 217878 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10639d079e40004, likely client has closed socket
   [junit4]   2> 217900 INFO  (zkConnectionManagerCallback-140-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 217908 INFO  (zkConnectionManagerCallback-142-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 218186 INFO  (jetty-launcher-123-thread-1) [n:127.0.0.1:54410_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 218187 INFO  (jetty-launcher-123-thread-1) [n:127.0.0.1:54410_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54410_solr
   [junit4]   2> 218188 INFO  (jetty-launcher-123-thread-2) [n:127.0.0.1:44548_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 218189 INFO  (jetty-launcher-123-thread-1) [n:127.0.0.1:54410_solr    ] o.a.s.c.Overseer Overseer (id=73810011459158022-127.0.0.1:54410_solr-n_0000000000) starting
   [junit4]   2> 218190 INFO  (jetty-launcher-123-thread-2) [n:127.0.0.1:44548_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44548_solr
   [junit4]   2> 218230 INFO  (zkCallback-139-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 218349 INFO  (zkCallback-141-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 218350 INFO  (zkConnectionManagerCallback-151-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 218352 INFO  (jetty-launcher-123-thread-1) [n:127.0.0.1:54410_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 218354 INFO  (jetty-launcher-123-thread-1) [n:127.0.0.1:54410_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53192/solr ready
   [junit4]   2> 218362 INFO  (jetty-launcher-123-thread-1) [n:127.0.0.1:54410_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 218362 INFO  (jetty-launcher-123-thread-1) [n:127.0.0.1:54410_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54410_solr
   [junit4]   2> 218363 INFO  (zkCallback-139-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 218364 INFO  (zkCallback-141-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 218373 DEBUG (OverseerAutoScalingTriggerThread-73810011459158022-127.0.0.1:54410_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 218391 DEBUG (OverseerAutoScalingTriggerThread-73810011459158022-127.0.0.1:54410_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 218392 DEBUG (OverseerAutoScalingTriggerThread-73810011459158022-127.0.0.1:54410_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 218392 DEBUG (OverseerAutoScalingTriggerThread-73810011459158022-127.0.0.1:54410_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 218472 INFO  (zkCallback-150-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 218499 DEBUG (OverseerAutoScalingTriggerThread-73810011459158022-127.0.0.1:54410_solr-n_0000000000) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:54410_solr, 127.0.0.1:44548_solr]
   [junit4]   2> 218511 DEBUG (ScheduledTrigger-289-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 218524 DEBUG (OverseerAutoScalingTriggerThread-73810011459158022-127.0.0.1:54410_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 218524 DEBUG (OverseerAutoScalingTriggerThread-73810011459158022-127.0.0.1:54410_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 218914 INFO  (jetty-launcher-123-thread-2) [n:127.0.0.1:44548_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44548.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@84b01ca
   [junit4]   2> 218926 INFO  (jetty-launcher-123-thread-2) [n:127.0.0.1:44548_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44548.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@84b01ca
   [junit4]   2> 218926 INFO  (jetty-launcher-123-thread-2) [n:127.0.0.1:44548_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44548.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@84b01ca
   [junit4]   2> 218928 INFO  (jetty-launcher-123-thread-2) [n:127.0.0.1:44548_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_633326200D19E9FD-001/tempDir-001/node2/.
   [junit4]   2> 219013 INFO  (zkConnectionManagerCallback-156-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 219015 INFO  (jetty-launcher-123-thread-2) [n:127.0.0.1:44548_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 219023 INFO  (jetty-launcher-123-thread-2) [n:127.0.0.1:44548_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53192/solr ready
   [junit4]   2> 219316 INFO  (jetty-launcher-123-thread-1) [n:127.0.0.1:54410_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54410.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@84b01ca
   [junit4]   2> 219329 INFO  (jetty-launcher-123-thread-1) [n:127.0.0.1:54410_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54410.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@84b01ca
   [junit4]   2> 219330 INFO  (jetty-launcher-123-thread-1) [n:127.0.0.1:54410_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54410.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@84b01ca
   [junit4]   2> 219331 INFO  (jetty-launcher-123-thread-1) [n:127.0.0.1:54410_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_633326200D19E9FD-001/tempDir-001/node1/.
   [junit4]   2> 219369 INFO  (zkConnectionManagerCallback-158-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 219384 INFO  (zkConnectionManagerCallback-163-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 219385 INFO  (SUITE-IndexSizeTriggerTest-seed#[633326200D19E9FD]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 219387 INFO  (SUITE-IndexSizeTriggerTest-seed#[633326200D19E9FD]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53192/solr ready
   [junit4]   2> 219415 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 219415 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 219419 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 219420 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 219434 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 219435 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 219436 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 219437 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219457 DEBUG (ScheduledTrigger-298-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219480 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219500 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219514 DEBUG (ScheduledTrigger-289-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219521 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219541 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219746 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219766 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219776 INFO  (TEST-IndexSizeTriggerTest.testMergeIntegration-seed#[633326200D19E9FD]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMergeIntegration
   [junit4]   2> 219787 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219807 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219828 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219848 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219868 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219888 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219910 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219930 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219948 DEBUG (simCloudManagerPool-297-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testMergeIntegration_collection / shard1: {"core_node1":{
   [junit4]   2>     "core":"testMergeIntegration_collection_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testMergeIntegration_collection"}}
   [junit4]   2> 219950 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219971 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 219991 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220011 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220032 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220052 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220073 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220093 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220114 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220134 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220154 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220180 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220200 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220220 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220241 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220261 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220281 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220289 DEBUG (simCloudManagerPool-297-thread-19) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 220289 DEBUG (simCloudManagerPool-297-thread-19) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 220292 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 220293 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 220293 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 220302 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220322 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220326 DEBUG (simCloudManagerPool-297-thread-20) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 220326 DEBUG (simCloudManagerPool-297-thread-20) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 220327 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 220328 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 220328 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 220337 DEBUG (simCloudManagerPool-297-thread-21) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 220337 DEBUG (simCloudManagerPool-297-thread-21) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 220341 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 220342 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 220342 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 220342 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220362 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220383 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220403 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220424 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220444 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220453 DEBUG (simCloudManagerPool-297-thread-28) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 220453 DEBUG (simCloudManagerPool-297-thread-28) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 220454 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 220454 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 220454 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 220465 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220485 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220505 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220514 DEBUG (ScheduledTrigger-289-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220526 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220546 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220698 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220741 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220762 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220782 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220802 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220823 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220843 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220863 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220883 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220904 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220924 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220944 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220965 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 220985 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221005 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221025 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221046 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221066 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221086 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221106 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221127 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221147 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221167 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221187 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221208 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221228 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221248 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221268 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221289 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221309 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221329 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221349 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221370 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221390 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221410 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221430 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221451 DEBUG (ScheduledTrigger-298-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221471 DEBUG (ScheduledTrigger-298-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221491 DEBUG (ScheduledTrigger-298-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221511 DEBUG (ScheduledTrigger-298-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221514 DEBUG (ScheduledTrigger-289-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221531 DEBUG (ScheduledTrigger-298-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221552 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221741 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221762 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221782 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221802 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221822 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221842 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221863 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221883 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221903 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221924 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221944 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221964 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 221985 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222005 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222025 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222046 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222066 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222086 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222107 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222127 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222147 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222241 DEBUG (ScheduledTrigger-298-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222262 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222282 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222302 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222322 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222343 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222359 INFO  (TEST-IndexSizeTriggerTest.testMergeIntegration-seed#[633326200D19E9FD]) [    ] o.a.s.c.a.IndexSizeTriggerTest #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes: 2
   [junit4]   2> ## Empty nodes: 0
   [junit4]   2> ## Dead nodes: 0
   [junit4]   2> ## Collections: [testMergeIntegration_collection]
   [junit4]   2> DocCollection(testMergeIntegration_collection//clusterstate.json/19)={
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"2",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{
   [junit4]   2>     "shard2":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testMergeIntegration_collection_shard2_replica_n3",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":3,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":3},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testMergeIntegration_collection_shard2_replica_n4",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":3,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":3}},
   [junit4]   2>       "range":"0-7fffffff",
   [junit4]   2>       "state":"active"},
   [junit4]   2>     "shard1":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node1":{
   [junit4]   2>           "core":"testMergeIntegration_collection_shard1_replica_n1",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":7,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":5,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":2},
   [junit4]   2>         "core_node2":{
   [junit4]   2>           "core":"testMergeIntegration_collection_shard1_replica_n2",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":7,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":5,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":2}},
   [junit4]   2>       "range":"80000000-ffffffff",
   [junit4]   2>       "state":"active"}}}
   [junit4]   2> ## Max replicas per node: 2
   [junit4]   2> ## Min replicas per node: 2
   [junit4]   2> ## Total replicas: 4
   [junit4]   2> ## * testMergeIntegration_collection 4
   [junit4]   2> ## - active           4
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ## - CREATE             1
   [junit4]   2> ## - autoscaling        4
   [junit4]   2> ## - update            17
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2>
   [junit4]   2> 222363 DEBUG (ScheduledTrigger-298-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222364 DEBUG (TEST-IndexSizeTriggerTest.testMergeIntegration-seed#[633326200D19E9FD]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 222364 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 222364 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 222364 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 222364 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 222465 INFO  (TEST-IndexSizeTriggerTest.testMergeIntegration-seed#[633326200D19E9FD]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testMergeIntegration
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=IndexSizeTriggerTest -Dtests.method=testMergeIntegration -Dtests.seed=633326200D19E9FD -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=nl-BE -Dtests.timezone=America/Argentina/Rio_Gallegos -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 2.73s J2 | IndexSizeTriggerTest.testMergeIntegration <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: did not finish processing in time
   [junit4]    > at __randomizedtesting.SeedInfo.seed([633326200D19E9FD:308A6490EF087C07]:0)
   [junit4]    > at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration(IndexSizeTriggerTest.java:425)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 222516 DEBUG (ScheduledTrigger-289-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 222531 INFO  (TEST-IndexSizeTriggerTest.testSplitIntegration-seed#[633326200D19E9FD]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSplitIntegration
   [junit4]   2> 222673 DEBUG (simCloudManagerPool-297-thread-34) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard2: {"core_node3":{
   [junit4]   2>     "core":"testSplitIntegration_collection_shard2_replica_n3",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testSplitIntegration_collection"}}
   [junit4]   2> 222678 DEBUG (simCloudManagerPool-297-thread-35) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard1: {"core_node1":{
   [junit4]   2>     "core":"testSplitIntegration_collection_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testSplitIntegration_collection"}}
   [junit4]   2> 222690 DEBUG (simCloudManagerPool-297-thread-36) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 222694 DEBUG (simCloudManagerPool-297-thread-36) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 222694 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 222694 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 222694 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 222694 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 222695 DEBUG (simCloudManagerPool-297-thread-37) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 222695 DEBUG (simCloudManagerPool-297-thread-37) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
   [junit4]   2> 222695 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 222695 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 222695 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 222695 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 222701 DEBUG (simCloudManagerPool-297-thread-38) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 222701 DEBUG (simCloudManagerPool-297-thread-38) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
   [junit4]   2> 222703 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
   [junit4]   2> 222703 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 222703 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 222703 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
   [junit4]   2> 223001 DEBUG (simCloudManagerPool-297-thread-90) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 223001 DEBUG (simCloudManagerPool-297-thread-90) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
   [junit4]   2> 223001 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
   [junit4]   2> 223002 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 223002 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 223002 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
   [junit4]   2> 223171 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"5effd7da54be83Tdz12trydta1f9960kx3i6dkzm",
   [junit4]   2>   "source":"index_size_trigger2",
   [junit4]   2>   "eventTime":26739950356840067,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}
   [junit4]   2> 223173 DEBUG (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [index_size_trigger2]
   [junit4]   2> 223175 INFO  (ScheduledTrigger-298-thread-4) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=26739958992684367, stage=STARTED, actionName='null', event={
   [junit4]   2>   "id":"5effd7da54be83Tdz12trydta1f9960kx3i6dkzm",
   [junit4]   2>   "source":"index_size_trigger2",
   [junit4]   2>   "eventTime":26739950356840067,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":26739958929590817,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}, context={}, config={
   [junit4]   2>   "trigger":"index_size_trigger2",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 223228 DEBUG (AutoscalingActionExecutor-299-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"5effd7da54be83Tdz12trydta1f9960kx3i6dkzm",
   [junit4]   2>   "source":"index_size_trigger2",
   [junit4]   2>   "eventTime":26739950356840067,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":26739958929590817,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}
   [junit4]   2> 223233 INFO  (AutoscalingActionExecutor-299-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=26739961904468967, stage=BEFORE_ACTION, actionName='compute_plan', event={
   [junit4]   2>   "id":"5effd7da54be83Tdz12trydta1f9960kx3i6dkzm",
   [junit4]   2>   "source":"index_size_trigger2",
   [junit4]   2>   "eventTime":26739950356840067,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":26739958929590817,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
   [junit4]   2>   "trigger":"index_size_trigger2",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 223234 DEBUG (AutoscalingActionExecutor-299-thread-1) [    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"5effd7da54be83Tdz12trydta1f9960kx3i6dkzm",
   [junit4]   2>   "source":"index_size_trigger2",
   [junit4]   2>   "eventTime":26739950356840067,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":26739958929590817,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
   [junit4]   2> 223236 INFO  (AutoscalingActionExecutor-299-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=SPLITSHARD&collection=testSplitIntegration_collection&shard=shard1
   [junit4]   2> 223237 INFO  (AutoscalingActionExecutor-299-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=SPLITSHARD&collection=testSplitIntegration_collection&shard=shard2
   [junit4]   2> 223237 INFO  (AutoscalingActionExecutor-299-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 26739962115 sessionWrapper.createTime 26739961982887017, this.sessionWrapper.createTime 26739961982887017
   [junit4]   2> 223238 INFO  (AutoscalingActionExecutor-299-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=26739962133984817, stage=AFTER_ACTION, actionName='compute_plan', event={
   [junit4]   2>   "id":"5effd7da54be83Tdz12trydta1f9960kx3i6dkzm",
   [junit4]   2>   "source":"index_size_trigger2",
   [junit4]   2>   "eventTime":26739950356840067,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":2,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":26739958929590817,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard2}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
   [junit4]   2>   "trigger":"index_size_trigger2",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 223238 INFO  (AutoscalingActionExecutor-299-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=26739962164160567, stage=BEFORE_ACTION, actionName='execute_plan', event={
   [junit4]   2>   "id":"5effd7da54be83Tdz12trydta1f9960kx3i6dkzm",
   [junit4]   2>   "source":"index_size_trigger2",
   [junit4]   2>   "eventTime":26739950356840067,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":2,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":26739958929590817,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard2}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
   [junit4]   2>   "trigger":"index_size_trigger2",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 223239 DEBUG (AutoscalingActionExecutor-299-thread-1) [    ] o.a.s.c.a.ExecutePlanAction -- processing event: {
   [junit4]   2>   "id":"5effd7da54be83Tdz12trydta1f9960kx3i6dkzm",
   [junit4]   2>   "source":"index_size_trigger2",
   [junit4]   2>   "eventTime":26739950356840067,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":2,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":26739958929590817,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard@75686d0f, org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard@6fcbdac4], AFTER_ACTION=[compute_plan], BEFORE_ACTION=[compute_plan, execute_plan]}
   [junit4]   2> 223239 DEBUG (AutoscalingActionExecutor-299-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=SPLITSHARD&collection=testSplitIntegration_collection&shard=shard1
   [junit4]   2> 223516 DEBUG (ScheduledTrigger-289-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 223550 DEBUG (simCloudManagerPool-297-thread-95) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard1_1: {"core_node10":{
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":12,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10000/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":12,
   [junit4]   2>     "core":"testSplitIntegration_collection_shard1_1_replica1",
   [junit4]   2>     "shard":"shard1_1",
   [junit4]   2>     "collection":"testSplitIntegration_collection"}}
   [junit4]   2> 223574 DEBUG (simCloudManagerPool-297-thread-96) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard1_0: {"core_node7":{
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":13,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10001/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":13,
   [junit4]   2>     "core":"testSplitIntegration_collection_shard1_0_replica0",
   [junit4]   2>     "shard":"shard1_0",
   [junit4]   2>     "collection":"testSplitIntegration_collection"}}
   [junit4]   2> 223600 DEBUG (AutoscalingActionExecutor-299-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=SPLITSHARD&collection=testSplitIntegration_collection&shard=shard2
   [junit4]   2> 223780 DEBUG (simCloudManagerPool-297-thread-102) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard2_1: {"core_node16":{
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":12,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10000/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":12,
   [junit4]   2>     "core":"testSplitIntegration_collection_shard2_1_replica1",
   [junit4]   2>     "shard":"shard2_1",
   [junit4]   2>     "collection":"testSplitIntegration_collection"}}
   [junit4]   2> 223780 DEBUG (simCloudManagerPool-297-thread-105) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard2_0: {"core_node14":{
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":13,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10000/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":13,
   [junit4]   2>     "core":"testSplitIntegration_collection_shard2_0_replica1",
   [junit4]   2>     "shard":"shard2_0",
   [junit4]   2>     "collection":"testSplitIntegration_collection"}}
   [junit4]   2> 223796 INFO  (AutoscalingActionExecutor-299-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=26739990045943567, stage=AFTER_ACTION, actionName='execute_plan', event={
   [junit4]   2>   "id":"5effd7da54be83Tdz12trydta1f9960kx3i6dkzm",
   [junit4]   2>   "source":"index_size_trigger2",
   [junit4]   2>   "eventTime":26739950356840067,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":2,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":26739958929590817,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/5effd7da54be83Tdz12trydta1f9960kx3i6dkzm/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/5effd7da54be83Tdz12trydta1f9960kx3i6dkzm/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard2}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/5effd7da54be83Tdz12trydta1f9960kx3i6dkzm/0] in completed tasks},success=}, {status={state=completed,msg=found [index_size_trigger2/5effd7da54be83Tdz12trydta1f9960kx3i6dkzm/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={
   [junit4]   2>   "trigger":"index_size_trigger2",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 223797 INFO  (AutoscalingActionExecutor-299-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=26739990103470367, stage=SUCCEEDED, actionName='null', event={
   [junit4]   2>   "id":"5effd7da54be83Tdz12trydta1f9960kx3i6dkzm",
   [junit4]   2>   "source":"index_size_trigger2",
   [junit4]   2>   "eventTime":26739950356840067,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":2,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":26739958929590817,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_coll

[...truncated too long message...]

pace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null1208031599
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null1208031599
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null1208031599

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null394833958
     [copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null394833958
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null394833958

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 42ms :: artifacts dl 2ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 157 minutes 43 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2531 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2531/

3 tests failed.
FAILED:  org.apache.solr.analytics.NoFacetTest.meanTest

Error Message:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:42031/solr/collection1, http://127.0.0.1:41607/solr/collection1]

Stack Trace:
java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:42031/solr/collection1, http://127.0.0.1:41607/solr/collection1]
        at __randomizedtesting.SeedInfo.seed([AE133F7198F12532:A495EFD4A603AA4D]:0)
        at org.apache.solr.analytics.SolrAnalyticsTestCase.queryCloudObject(SolrAnalyticsTestCase.java:147)
        at org.apache.solr.analytics.SolrAnalyticsTestCase.testResults(SolrAnalyticsTestCase.java:98)
        at org.apache.solr.analytics.SolrAnalyticsTestCase.testAnalytics(SolrAnalyticsTestCase.java:162)
        at org.apache.solr.analytics.SolrAnalyticsTestCase.testExpressions(SolrAnalyticsTestCase.java:181)
        at org.apache.solr.analytics.NoFacetTest.meanTest(NoFacetTest.java:172)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:42031/solr/collection1, http://127.0.0.1:41607/solr/collection1]
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.analytics.SolrAnalyticsTestCase.queryCloudObject(SolrAnalyticsTestCase.java:145)
        ... 43 more
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41607/solr/collection1: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        ... 48 more


FAILED:  org.apache.solr.analytics.NoFacetTest.medianTest

Error Message:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:41607/solr/collection1]

Stack Trace:
java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:41607/solr/collection1]
        at __randomizedtesting.SeedInfo.seed([AE133F7198F12532:15BC4528A578261]:0)
        at org.apache.solr.analytics.SolrAnalyticsTestCase.queryCloudObject(SolrAnalyticsTestCase.java:147)
        at org.apache.solr.analytics.SolrAnalyticsTestCase.testResults(SolrAnalyticsTestCase.java:98)
        at org.apache.solr.analytics.SolrAnalyticsTestCase.testAnalytics(SolrAnalyticsTestCase.java:162)
        at org.apache.solr.analytics.SolrAnalyticsTestCase.testExpressions(SolrAnalyticsTestCase.java:181)
        at org.apache.solr.analytics.NoFacetTest.medianTest(NoFacetTest.java:220)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:41607/solr/collection1]
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:993)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.analytics.SolrAnalyticsTestCase.queryCloudObject(SolrAnalyticsTestCase.java:145)
        ... 43 more
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41607/solr/collection1: no servers hosting shard: shard1
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:436)
        ... 49 more


FAILED:  org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure

Error Message:
Did not see a fully active cluster after 30 seconds

Stack Trace:
java.lang.AssertionError: Did not see a fully active cluster after 30 seconds
        at __randomizedtesting.SeedInfo.seed([9BB942761B40B0A2:138FE025C3EF58B0]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure(TestCollectionStateWatchers.java:250)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 16225 lines...]
   [junit4] Suite: org.apache.solr.common.cloud.TestCollectionStateWatchers
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_9BB942761B40B0A2-001/init-core-data-001
   [junit4]   2> 658096 WARN  (SUITE-TestCollectionStateWatchers-seed#[9BB942761B40B0A2]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=35 numCloses=35
   [junit4]   2> 658112 INFO  (SUITE-TestCollectionStateWatchers-seed#[9BB942761B40B0A2]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 658113 INFO  (SUITE-TestCollectionStateWatchers-seed#[9BB942761B40B0A2]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 658113 INFO  (SUITE-TestCollectionStateWatchers-seed#[9BB942761B40B0A2]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 658114 INFO  (SUITE-TestCollectionStateWatchers-seed#[9BB942761B40B0A2]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_9BB942761B40B0A2-001/tempDir-001
   [junit4]   2> 658114 INFO  (SUITE-TestCollectionStateWatchers-seed#[9BB942761B40B0A2]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 658149 INFO  (Thread-609) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 658149 INFO  (Thread-609) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 658151 ERROR (Thread-609) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 658242 INFO  (SUITE-TestCollectionStateWatchers-seed#[9BB942761B40B0A2]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:37111
   [junit4]   2> 658311 INFO  (zkConnectionManagerCallback-721-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 658349 INFO  (jetty-launcher-718-thread-1) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 658369 INFO  (jetty-launcher-718-thread-2) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 658402 INFO  (jetty-launcher-718-thread-3) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 658442 INFO  (jetty-launcher-718-thread-4) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 658741 INFO  (jetty-launcher-718-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 658741 INFO  (jetty-launcher-718-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 658741 INFO  (jetty-launcher-718-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 658750 INFO  (jetty-launcher-718-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 658750 INFO  (jetty-launcher-718-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 658750 INFO  (jetty-launcher-718-thread-4) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 658758 INFO  (jetty-launcher-718-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@34d025e7{/solr,null,AVAILABLE}
   [junit4]   2> 658759 INFO  (jetty-launcher-718-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 658759 INFO  (jetty-launcher-718-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 658759 INFO  (jetty-launcher-718-thread-2) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 658770 INFO  (jetty-launcher-718-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 658770 INFO  (jetty-launcher-718-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 658771 INFO  (jetty-launcher-718-thread-3) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 658774 INFO  (jetty-launcher-718-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7fc01711{/solr,null,AVAILABLE}
   [junit4]   2> 658831 INFO  (jetty-launcher-718-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@af4f3d8{/solr,null,AVAILABLE}
   [junit4]   2> 658832 INFO  (jetty-launcher-718-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@58ff5804{/solr,null,AVAILABLE}
   [junit4]   2> 658834 INFO  (jetty-launcher-718-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2407f6d7{HTTP/1.1,[http/1.1]}{127.0.0.1:46074}
   [junit4]   2> 658834 INFO  (jetty-launcher-718-thread-3) [    ] o.e.j.s.Server Started @659007ms
   [junit4]   2> 658834 INFO  (jetty-launcher-718-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46074}
   [junit4]   2> 658834 ERROR (jetty-launcher-718-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 658834 INFO  (jetty-launcher-718-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 658834 INFO  (jetty-launcher-718-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 658834 INFO  (jetty-launcher-718-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 658834 INFO  (jetty-launcher-718-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 658834 INFO  (jetty-launcher-718-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-17T04:46:40.224Z
   [junit4]   2> 658835 INFO  (jetty-launcher-718-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4c3e5479{HTTP/1.1,[http/1.1]}{127.0.0.1:37652}
   [junit4]   2> 658835 INFO  (jetty-launcher-718-thread-2) [    ] o.e.j.s.Server Started @659008ms
   [junit4]   2> 658835 INFO  (jetty-launcher-718-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37652}
   [junit4]   2> 658848 INFO  (jetty-launcher-718-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@540d6e09{HTTP/1.1,[http/1.1]}{127.0.0.1:43619}
   [junit4]   2> 658848 INFO  (jetty-launcher-718-thread-1) [    ] o.e.j.s.Server Started @659021ms
   [junit4]   2> 658848 INFO  (jetty-launcher-718-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43619}
   [junit4]   2> 658848 ERROR (jetty-launcher-718-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 658848 INFO  (jetty-launcher-718-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 658848 INFO  (jetty-launcher-718-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 658848 INFO  (jetty-launcher-718-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 658848 INFO  (jetty-launcher-718-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 658848 INFO  (jetty-launcher-718-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-17T04:46:40.238Z
   [junit4]   2> 658849 INFO  (jetty-launcher-718-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@c6aae1c{HTTP/1.1,[http/1.1]}{127.0.0.1:35409}
   [junit4]   2> 658849 INFO  (jetty-launcher-718-thread-4) [    ] o.e.j.s.Server Started @659023ms
   [junit4]   2> 658849 INFO  (jetty-launcher-718-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35409}
   [junit4]   2> 658850 ERROR (jetty-launcher-718-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 658850 INFO  (jetty-launcher-718-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 658850 INFO  (jetty-launcher-718-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 658850 INFO  (jetty-launcher-718-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 658850 INFO  (jetty-launcher-718-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 658850 INFO  (jetty-launcher-718-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-17T04:46:40.240Z
   [junit4]   2> 658850 ERROR (jetty-launcher-718-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 658850 INFO  (jetty-launcher-718-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 658850 INFO  (jetty-launcher-718-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 658851 INFO  (jetty-launcher-718-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 658851 INFO  (jetty-launcher-718-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 658851 INFO  (jetty-launcher-718-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-17T04:46:40.241Z
   [junit4]   2> 658953 INFO  (zkConnectionManagerCallback-723-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 658962 INFO  (zkConnectionManagerCallback-727-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 658963 INFO  (jetty-launcher-718-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 658974 INFO  (zkConnectionManagerCallback-725-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 658974 INFO  (jetty-launcher-718-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 658994 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10210ae6a1d0001, likely client has closed socket
   [junit4]   2> 659010 INFO  (zkConnectionManagerCallback-729-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 659026 INFO  (jetty-launcher-718-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 659026 INFO  (jetty-launcher-718-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 659030 INFO  (jetty-launcher-718-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37111/solr
   [junit4]   2> 659081 INFO  (jetty-launcher-718-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37111/solr
   [junit4]   2> 659082 INFO  (zkConnectionManagerCallback-737-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 659084 INFO  (jetty-launcher-718-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37111/solr
   [junit4]   2> 659132 INFO  (zkConnectionManagerCallback-734-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 659157 INFO  (zkConnectionManagerCallback-743-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 659185 INFO  (zkConnectionManagerCallback-741-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 659312 WARN  (jetty-launcher-718-thread-3) [n:127.0.0.1:46074_solr    ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
   [junit4]   2> 659320 INFO  (jetty-launcher-718-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37111/solr
   [junit4]   2> 659384 INFO  (zkConnectionManagerCallback-748-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 659385 INFO  (zkConnectionManagerCallback-746-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 659405 INFO  (jetty-launcher-718-thread-3) [n:127.0.0.1:46074_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 659406 INFO  (jetty-launcher-718-thread-3) [n:127.0.0.1:46074_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46074_solr
   [junit4]   2> 659407 INFO  (zkConnectionManagerCallback-753-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 659407 INFO  (jetty-launcher-718-thread-3) [n:127.0.0.1:46074_solr    ] o.a.s.c.Overseer Overseer (id=72638885281988616-127.0.0.1:46074_solr-n_0000000000) starting
   [junit4]   2> 659408 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10210ae6a1d000b, likely client has closed socket
   [junit4]   2> 659442 INFO  (zkConnectionManagerCallback-755-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 659460 WARN  (jetty-launcher-718-thread-4) [n:127.0.0.1:35409_solr    ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
   [junit4]   2> 659495 WARN  (jetty-launcher-718-thread-1) [n:127.0.0.1:43619_solr    ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
   [junit4]   2> 659497 WARN  (jetty-launcher-718-thread-2) [n:127.0.0.1:37652_solr    ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
   [junit4]   2> 659568 INFO  (jetty-launcher-718-thread-4) [n:127.0.0.1:35409_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 659570 INFO  (jetty-launcher-718-thread-1) [n:127.0.0.1:43619_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 659570 INFO  (jetty-launcher-718-thread-4) [n:127.0.0.1:35409_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35409_solr
   [junit4]   2> 659605 INFO  (jetty-launcher-718-thread-1) [n:127.0.0.1:43619_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 659606 INFO  (jetty-launcher-718-thread-1) [n:127.0.0.1:43619_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43619_solr
   [junit4]   2> 659753 INFO  (zkCallback-745-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 659758 INFO  (zkCallback-742-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 659953 INFO  (zkCallback-754-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 660002 INFO  (zkConnectionManagerCallback-766-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 660087 INFO  (zkCallback-747-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 660160 INFO  (jetty-launcher-718-thread-2) [n:127.0.0.1:37652_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 660176 INFO  (jetty-launcher-718-thread-3) [n:127.0.0.1:46074_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 660249 INFO  (jetty-launcher-718-thread-3) [n:127.0.0.1:46074_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37111/solr ready
   [junit4]   2> 660263 INFO  (jetty-launcher-718-thread-3) [n:127.0.0.1:46074_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 660263 INFO  (jetty-launcher-718-thread-3) [n:127.0.0.1:46074_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46074_solr
   [junit4]   2> 660286 INFO  (zkCallback-745-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 660286 INFO  (zkCallback-747-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 660286 INFO  (zkCallback-754-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 660302 INFO  (zkCallback-742-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 660306 INFO  (jetty-launcher-718-thread-2) [n:127.0.0.1:37652_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 660306 INFO  (jetty-launcher-718-thread-2) [n:127.0.0.1:37652_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37652_solr
   [junit4]   2> 660477 INFO  (zkCallback-765-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 660535 INFO  (zkCallback-765-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 660535 INFO  (zkCallback-745-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 660542 INFO  (zkCallback-747-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 660542 INFO  (zkCallback-754-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 660558 INFO  (OverseerStateUpdate-72638885281988616-127.0.0.1:46074_solr-n_0000000000) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 661049 INFO  (jetty-launcher-718-thread-1) [n:127.0.0.1:43619_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43619.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3ef1f3
   [junit4]   2> 661138 INFO  (jetty-launcher-718-thread-1) [n:127.0.0.1:43619_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43619.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3ef1f3
   [junit4]   2> 661138 INFO  (jetty-launcher-718-thread-1) [n:127.0.0.1:43619_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43619.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3ef1f3
   [junit4]   2> 661139 INFO  (jetty-launcher-718-thread-1) [n:127.0.0.1:43619_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_9BB942761B40B0A2-001/tempDir-001/node1/.
   [junit4]   2> 661266 INFO  (zkConnectionManagerCallback-771-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 661267 INFO  (jetty-launcher-718-thread-1) [n:127.0.0.1:43619_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 661268 INFO  (jetty-launcher-718-thread-1) [n:127.0.0.1:43619_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37111/solr ready
   [junit4]   2> 662330 INFO  (jetty-launcher-718-thread-4) [n:127.0.0.1:35409_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35409.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3ef1f3
   [junit4]   2> 662346 INFO  (jetty-launcher-718-thread-3) [n:127.0.0.1:46074_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46074.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3ef1f3
   [junit4]   2> 662418 INFO  (jetty-launcher-718-thread-4) [n:127.0.0.1:35409_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35409.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3ef1f3
   [junit4]   2> 662418 INFO  (jetty-launcher-718-thread-4) [n:127.0.0.1:35409_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35409.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3ef1f3
   [junit4]   2> 662419 INFO  (jetty-launcher-718-thread-4) [n:127.0.0.1:35409_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_9BB942761B40B0A2-001/tempDir-001/node4/.
   [junit4]   2> 662456 INFO  (zkConnectionManagerCallback-776-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 662457 INFO  (jetty-launcher-718-thread-4) [n:127.0.0.1:35409_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 662458 INFO  (jetty-launcher-718-thread-4) [n:127.0.0.1:35409_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37111/solr ready
   [junit4]   2> 662471 INFO  (jetty-launcher-718-thread-3) [n:127.0.0.1:46074_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46074.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3ef1f3
   [junit4]   2> 662471 INFO  (jetty-launcher-718-thread-3) [n:127.0.0.1:46074_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46074.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3ef1f3
   [junit4]   2> 662472 INFO  (jetty-launcher-718-thread-3) [n:127.0.0.1:46074_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_9BB942761B40B0A2-001/tempDir-001/node3/.
   [junit4]   2> 663174 INFO  (jetty-launcher-718-thread-2) [n:127.0.0.1:37652_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37652.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3ef1f3
   [junit4]   2> 663210 INFO  (jetty-launcher-718-thread-2) [n:127.0.0.1:37652_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37652.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3ef1f3
   [junit4]   2> 663233 INFO  (jetty-launcher-718-thread-2) [n:127.0.0.1:37652_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37652.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3ef1f3
   [junit4]   2> 663242 INFO  (jetty-launcher-718-thread-2) [n:127.0.0.1:37652_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_9BB942761B40B0A2-001/tempDir-001/node2/.
   [junit4]   2> 663262 INFO  (zkConnectionManagerCallback-781-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 663263 INFO  (jetty-launcher-718-thread-2) [n:127.0.0.1:37652_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 663264 INFO  (jetty-launcher-718-thread-2) [n:127.0.0.1:37652_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37111/solr ready
   [junit4]   2> 663463 INFO  (zkConnectionManagerCallback-783-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 663476 INFO  (zkConnectionManagerCallback-788-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 663494 INFO  (SUITE-TestCollectionStateWatchers-seed#[9BB942761B40B0A2]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 663496 INFO  (SUITE-TestCollectionStateWatchers-seed#[9BB942761B40B0A2]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37111/solr ready
   [junit4]   2> 663852 INFO  (TEST-TestCollectionStateWatchers.testWatchesWorkForStateFormat1-seed#[9BB942761B40B0A2]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testWatchesWorkForStateFormat1
   [junit4]   2> 663867 INFO  (zkConnectionManagerCallback-790-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 664080 INFO  (qtp447947068-2799) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=cf353f2c-479b-41af-b283-3178a64b0529&replicationFactor=1&collection.configName=config&name=stateformat1&nrtReplicas=1&action=CREATE&numShards=1&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 664118 INFO  (qtp447947068-2799) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=cf353f2c-479b-41af-b283-3178a64b0529&replicationFactor=1&collection.configName=config&name=stateformat1&nrtReplicas=1&action=CREATE&numShards=1&stateFormat=1&wt=javabin&version=2} status=0 QTime=37
   [junit4]   2> 664168 INFO  (OverseerThreadFactory-804-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection stateformat1
   [junit4]   2> 664188 INFO  (qtp447947068-2796) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=cf353f2c-479b-41af-b283-3178a64b0529&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 664190 INFO  (qtp447947068-2796) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=cf353f2c-479b-41af-b283-3178a64b0529&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 664851 INFO  (OverseerStateUpdate-72638885281988616-127.0.0.1:46074_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"stateformat1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"stateformat1_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:46074/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 665056 WARN  (watches-785-thread-1) [    ] o.a.s.c.c.ZkStateReader Error on calling watcher
   [junit4]   2> java.lang.NullPointerException: null
   [junit4]   2> at java.util.TreeMap.getEntry(TreeMap.java:347) ~[?:1.8.0_172]
   [junit4]   2> at java.util.TreeMap.containsKey(TreeMap.java:232) ~[?:1.8.0_172]
   [junit4]   2> at java.util.TreeSet.contains(TreeSet.java:234) ~[?:1.8.0_172]
   [junit4]   2> at org.apache.solr.common.cloud.Replica.isActive(Replica.java:166) ~[java/:?]
   [junit4]   2> at org.apache.solr.common.cloud.DocCollection.isFullyActive(DocCollection.java:301) ~[java/:?]
   [junit4]   2> at org.apache.solr.common.cloud.TestCollectionStateWatchers.lambda$testWatchesWorkForStateFormat1$20(TestCollectionStateWatchers.java:287) ~[test/:?]
   [junit4]   2> at org.apache.solr.common.cloud.ZkStateReader.lambda$waitForState$9(ZkStateReader.java:1445) ~[java/:?]
   [junit4]   2> at org.apache.solr.common.cloud.ZkStateReader$Notification.run(ZkStateReader.java:1628) ~[java/:?]
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) ~[java/:?]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 665248 INFO  (qtp90714700-2805) [n:127.0.0.1:46074_solr    x:stateformat1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=config&newCollection=true&collection=stateformat1&version=2&replicaType=NRT&async=cf353f2c-479b-41af-b283-3178a64b05298869807175471369&coreNodeName=core_node2&name=stateformat1_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin} status=0 QTime=56
   [junit4]   2> 665249 INFO  (qtp447947068-2798) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=cf353f2c-479b-41af-b283-3178a64b0529&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 665250 INFO  (qtp90714700-2809) [n:127.0.0.1:46074_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : cf353f2c-479b-41af-b283-3178a64b05298869807175471369
   [junit4]   2> 665286 INFO  (qtp90714700-2809) [n:127.0.0.1:46074_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=cf353f2c-479b-41af-b283-3178a64b05298869807175471369&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=35
   [junit4]   2> 665287 INFO  (qtp447947068-2798) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=cf353f2c-479b-41af-b283-3178a64b0529&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=38
   [junit4]   2> 665289 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr    x:stateformat1_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=config&newCollection=true&collection=stateformat1&version=2&replicaType=NRT&async=cf353f2c-479b-41af-b283-3178a64b05298869807175471369&coreNodeName=core_node2&name=stateformat1_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin
   [junit4]   2> 666350 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=cf353f2c-479b-41af-b283-3178a64b0529&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 666350 INFO  (qtp90714700-2806) [n:127.0.0.1:46074_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : cf353f2c-479b-41af-b283-3178a64b05298869807175471369
   [junit4]   2> 666351 INFO  (qtp90714700-2806) [n:127.0.0.1:46074_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=cf353f2c-479b-41af-b283-3178a64b05298869807175471369&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 666352 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=cf353f2c-479b-41af-b283-3178a64b0529&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 666519 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 666570 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.s.IndexSchema [stateformat1_shard1_replica_n1] Schema name=test
   [junit4]   2> 666958 WARN  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 666958 WARN  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 666977 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 666977 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'stateformat1_shard1_replica_n1' using configuration from collection stateformat1, trusted=true
   [junit4]   2> 666977 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46074.solr.core.stateformat1.shard1.replica_n1' (registry 'solr.core.stateformat1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3ef1f3
   [junit4]   2> 666977 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 666978 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.c.SolrCore [[stateformat1_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_9BB942761B40B0A2-001/tempDir-001/node3/stateformat1_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_9BB942761B40B0A2-001/tempDir-001/node3/./stateformat1_shard1_replica_n1/data/]
   [junit4]   2> 667206 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 667206 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 667223 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 667224 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 667225 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1d2bfd2b[stateformat1_shard1_replica_n1] main]
   [junit4]   2> 667226 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
   [junit4]   2> 667226 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 667239 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 667240 INFO  (searcherExecutor-813-thread-1-processing-x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 r:core_node2 CREATE n:127.0.0.1:46074_solr c:stateformat1 s:shard1) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.c.SolrCore [stateformat1_shard1_replica_n1] Registered new searcher Searcher@1d2bfd2b[stateformat1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 667240 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600685246911610880
   [junit4]   2> 667274 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/stateformat1/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 667276 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 667277 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 667277 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46074/solr/stateformat1_shard1_replica_n1/
   [junit4]   2> 667277 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 667277 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:46074/solr/stateformat1_shard1_replica_n1/ has no replicas
   [junit4]   2> 667277 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 667297 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46074/solr/stateformat1_shard1_replica_n1/ shard1
   [junit4]   2> 667365 INFO  (qtp447947068-2798) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=cf353f2c-479b-41af-b283-3178a64b0529&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 667385 INFO  (qtp90714700-2806) [n:127.0.0.1:46074_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : cf353f2c-479b-41af-b283-3178a64b05298869807175471369
   [junit4]   2> 667385 INFO  (qtp90714700-2806) [n:127.0.0.1:46074_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=cf353f2c-479b-41af-b283-3178a64b05298869807175471369&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=33
   [junit4]   2> 667423 INFO  (qtp447947068-2798) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=cf353f2c-479b-41af-b283-3178a64b0529&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=57
   [junit4]   2> 667466 INFO  (parallelCoreAdminExecutor-807-thread-1-processing-n:127.0.0.1:46074_solr x:stateformat1_shard1_replica_n1 cf353f2c-479b-41af-b283-3178a64b05298869807175471369 CREATE) [n:127.0.0.1:46074_solr c:stateformat1 s:shard1 r:core_node2 x:stateformat1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 668390 INFO  (qtp90714700-2801) [n:127.0.0.1:46074_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : cf353f2c-479b-41af-b283-3178a64b05298869807175471369
   [junit4]   2> 668390 INFO  (qtp90714700-2801) [n:127.0.0.1:46074_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=cf353f2c-479b-41af-b283-3178a64b05298869807175471369&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 668416 INFO  (OverseerThreadFactory-804-thread-1) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 668424 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=cf353f2c-479b-41af-b283-3178a64b0529&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 668456 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=cf353f2c-479b-41af-b283-3178a64b0529&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=32
   [junit4]   2> 668457 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=cf353f2c-479b-41af-b283-3178a64b0529&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 668487 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=cf353f2c-479b-41af-b283-3178a64b0529&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=30
   [junit4]   2> 668490 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :migratestateformat with params async=14266e93-454e-48b1-bf68-5da5a840c4ff&action=MIGRATESTATEFORMAT&collection=stateformat1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 668493 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr c:stateformat1   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=14266e93-454e-48b1-bf68-5da5a840c4ff&action=MIGRATESTATEFORMAT&collection=stateformat1&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 668494 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=14266e93-454e-48b1-bf68-5da5a840c4ff&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 668511 INFO  (OverseerCollectionConfigSetProcessor-72638885281988616-127.0.0.1:46074_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 668512 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=14266e93-454e-48b1-bf68-5da5a840c4ff&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=18
   [junit4]   2> 668614 INFO  (zkCallback-787-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/stateformat1/state.json] for collection [stateformat1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 668614 INFO  (zkCallback-742-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/stateformat1/state.json] for collection [stateformat1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 668615 INFO  (zkCallback-742-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/stateformat1/state.json] for collection [stateformat1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 668615 INFO  (zkCallback-787-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/stateformat1/state.json] for collection [stateformat1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 668617 INFO  (zkCallback-742-thread-3) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/stateformat1/state.json] for collection [stateformat1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 668725 INFO  (OverseerThreadFactory-804-thread-2) [ c:stateformat1   ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000001 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 669513 INFO  (qtp447947068-2798) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=14266e93-454e-48b1-bf68-5da5a840c4ff&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 669514 INFO  (qtp447947068-2798) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=14266e93-454e-48b1-bf68-5da5a840c4ff&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 669515 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=14266e93-454e-48b1-bf68-5da5a840c4ff&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 669516 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=14266e93-454e-48b1-bf68-5da5a840c4ff&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 669516 INFO  (TEST-TestCollectionStateWatchers.testWatchesWorkForStateFormat1-seed#[9BB942761B40B0A2]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testWatchesWorkForStateFormat1
   [junit4]   2> 669643 INFO  (TEST-TestCollectionStateWatchers.testPredicateFailureTimesOut-seed#[9BB942761B40B0A2]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testPredicateFailureTimesOut
   [junit4]   2> 669653 INFO  (zkConnectionManagerCallback-792-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 670516 INFO  (OverseerCollectionConfigSetProcessor-72638885281988616-127.0.0.1:46074_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000001 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 670677 INFO  (TEST-TestCollectionStateWatchers.testPredicateFailureTimesOut-seed#[9BB942761B40B0A2]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testPredicateFailureTimesOut
   [junit4]   2> 670886 INFO  (TEST-TestCollectionStateWatchers.testSimpleCollectionWatch-seed#[9BB942761B40B0A2]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSimpleCollectionWatch
   [junit4]   2> 671009 INFO  (zkConnectionManagerCallback-794-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 671030 INFO  (qtp447947068-2798) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=da4723d5-29e8-4413-a7f0-19be46ddf774&replicationFactor=1&collection.configName=config&name=testcollection&nrtReplicas=1&action=CREATE&numShards=4&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 671084 INFO  (qtp447947068-2798) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=da4723d5-29e8-4413-a7f0-19be46ddf774&replicationFactor=1&collection.configName=config&name=testcollection&nrtReplicas=1&action=CREATE&numShards=4&wt=javabin&version=2} status=0 QTime=54
   [junit4]   2> 671085 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=da4723d5-29e8-4413-a7f0-19be46ddf774&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 671105 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=da4723d5-29e8-4413-a7f0-19be46ddf774&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=20
   [junit4]   2> 671117 INFO  (OverseerThreadFactory-804-thread-3) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection testcollection
   [junit4]   2> 671334 INFO  (OverseerStateUpdate-72638885281988616-127.0.0.1:46074_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:37652/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 671385 INFO  (OverseerStateUpdate-72638885281988616-127.0.0.1:46074_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:46074/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 671386 INFO  (OverseerStateUpdate-72638885281988616-127.0.0.1:46074_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard3",
   [junit4]   2>   "core":"testcollection_shard3_replica_n3",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:35409/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 671405 INFO  (OverseerStateUpdate-72638885281988616-127.0.0.1:46074_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard4",
   [junit4]   2>   "core":"testcollection_shard4_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:43619/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 671642 INFO  (qtp499128381-2810) [n:127.0.0.1:37652_solr    x:testcollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=config&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=da4723d5-29e8-4413-a7f0-19be46ddf7748869813660738003&coreNodeName=core_node5&name=testcollection_shard1_replica_n1&action=CREATE&numShards=4&shard=shard1&wt=javabin} status=0 QTime=36
   [junit4]   2> 671776 INFO  (qtp447947068-2794) [n:127.0.0.1:35409_solr    x:testcollection_shard3_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=config&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=da4723d5-29e8-4413-a7f0-19be46ddf7748869813695690926&coreNodeName=core_node7&name=testcollection_shard3_replica_n3&action=CREATE&numShards=4&shard=shard3&wt=javabin} status=0 QTime=158
   [junit4]   2> 671861 INFO  (parallelCoreAdminExecutor-811-thread-1-processing-n:127.0.0.1:37652_solr x:testcollection_shard1_replica_n1 da4723d5-29e8-4413-a7f0-19be46ddf7748869813660738003 CREATE) [n:127.0.0.1:37652_solr    x:testcollection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=config&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=da4723d5-29e8-4413-a7f0-19be46ddf7748869813660738003&coreNodeName=core_node5&name=testcollection_shard1_replica_n1&action=CREATE&numShards=4&shard=shard1&wt=javabin
   [junit4]   2> 671885 INFO  (parallelCoreAdminExecutor-808-thread-1-processing-n:127.0.0.1:35409_solr x:testcollection_shard3_replica_n3 da4723d5-29e8-4413-a7f0-19be46ddf7748869813695690926 CREATE) [n:127.0.0.1:35409_solr    x:testcollection_shard3_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=config&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=da4723d5-29e8-4413-a7f0-19be46ddf7748869813695690926&coreNodeName=core_node7&name=testcollection_shard3_replica_n3&action=CREATE&numShards=4&shard=shard3&wt=javabin
   [junit4]   2> 671886 INFO  (parallelCoreAdminExecutor-808-thread-1-processing-n:127.0.0.1:35409_solr x:testcollection_shard3_replica_n3 da4723d5-29e8-4413-a7f0-19be46ddf7748869813695690926 CREATE) [n:127.0.0.1:35409_solr    x:testcollection_shard3_replica_n3] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 671910 INFO  (qtp435842460-2820) [n:127.0.0.1:43619_solr    x:testcollection_shard4_replica_n4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=config&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=da4723d5-29e8-4413-a7f0-19be46ddf7748869813697424344&coreNodeName=core_node8&name=testcollection_shard4_replica_n4&action=CREATE&numShards=4&shard=shard4&wt=javabin} status=0 QTime=12
   [junit4]   2> 671967 INFO  (parallelCoreAdminExecutor-805-thread-1-processing-n:127.0.0.1:43619_solr x:testcollection_shard4_replica_n4 da4723d5-29e8-4413-a7f0-19be46ddf7748869813697424344 CREATE) [n:127.0.0.1:43619_solr    x:testcollection_shard4_replica_n4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=config&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=da4723d5-29e8-4413-a7f0-19be46ddf7748869813697424344&coreNodeName=core_node8&name=testcollection_shard4_replica_n4&action=CREATE&numShards=4&shard=shard4&wt=javabin
   [junit4]   2> 672003 INFO  (qtp90714700-2806) [n:127.0.0.1:46074_solr    x:testcollection_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=config&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=da4723d5-29e8-4413-a7f0-19be46ddf7748869813676680554&coreNodeName=core_node6&name=testcollection_shard2_replica_n2&action=CREATE&numShards=4&shard=shard2&wt=javabin} status=0 QTime=381
   [junit4]   2> 672019 INFO  (parallelCoreAdminExecutor-807-thread-2-processing-n:127.0.0.1:46074_solr x:testcollection_shard2_replica_n2 da4723d5-29e8-4413-a7f0-19be46ddf7748869813676680554 CREATE) [n:127.0.0.1:46074_solr    x:testcollection_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=config&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=da4723d5-29e8-4413-a7f0-19be46ddf7748869813676680554&coreNodeName=core_node6&name=testcollection_shard2_replica_n2&action=CREATE&numShards=4&shard=shard2&wt=javabin
   [junit4]   2> 672106 INFO  (qtp447947068-2798) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=da4723d5-29e8-4413-a7f0-19be46ddf774&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 672182 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : da4723d5-29e8-4413-a7f0-19be46ddf7748869813695690926
   [junit4]   2> 672182 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=da4723d5-29e8-4413-a7f0-19be46ddf7748869813695690926&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 672266 INFO  (qtp447947068-2798) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=da4723d5-29e8-4413-a7f0-19be46ddf774&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=160
   [junit4]   2> 673028 INFO  (parallelCoreAdminExecutor-811-thread-1-processing-n:127.0.0.1:37652_solr x:testcollection_shard1_replica_n1 da4723d5-29e8-4413-a7f0-19be46ddf7748869813660738003 CREATE) [n:127.0.0.1:37652_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 673120 INFO  (parallelCoreAdminExecutor-811-thread-1-processing-n:127.0.0.1:37652_solr x:testcollection_shard1_replica_n1 da4723d5-29e8-4413-a7f0-19be46ddf7748869813660738003 CREATE) [n:127.0.0.1:37652_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema [testcollection_shard1_replica_n1] Schema name=test
   [junit4]   2> 673241 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : da4723d5-29e8-4413-a7f0-19be46ddf7748869813695690926
   [junit4]   2> 673241 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=da4723d5-29e8-4413-a7f0-19be46ddf7748869813695690926&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 673334 INFO  (qtp447947068-2802) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=da4723d5-29e8-4413-a7f0-19be46ddf774&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 673358 INFO  (parallelCoreAdminExecutor-805-thread-1-processing-n:127.0.0.1:43619_solr x:testcollection_shard4_replica_n4 da4723d5-29e8-4413-a7f0-19be46ddf7748869813697424344 CREATE) [n:127.0.0.1:43619_solr c:testcollection s:shard4 r:core_node8 x:testcollection_shard4_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 673372 INFO  (qtp447947068-2802) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=da4723d5-29e8-4413-a7f0-19be46ddf774&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=38
   [junit4]   2> 673388 INFO  (parallelCoreAdminExecutor-808-thread-1-processing-n:127.0.0.1:35409_solr x:testcollection_shard3_replica_n3 da4723d5-29e8-4413-a7f0-19be46ddf7748869813695690926 CREATE) [n:127.0.0.1:35409_solr c:testcollection s:shard3 r:core_node7 x:testcollection_shard3_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 673705 INFO  (parallelCoreAdminExecutor-805-thread-1-processing-n:127.0.0.1:43619_solr x:testcollection_shard4_replica_n4 da4723d5-29e8-4413-a7f0-19be46ddf7748869813697424344 CREATE) [n:127.0.0.1:43619_solr c:testcollection s:shard4 r:core_node8 x:testcollection_shard4_replica_n4] o.a.s.s.IndexSchema [testcollection_shard4_replica_n4] Schema name=test
   [junit4]   2> 673721 INFO  (parallelCoreAdminExecutor-808-thread-1-processing-n:127.0.0.1:35409_solr x:testcollection_shard3_replica_n3 da4723d5-29e8-4413-a7f0-19be46ddf7748869813695690926 CREATE) [n:127.0.0.1:35409_solr c:testcollection s:shard3 r:core_node7 x:testcollection_shard3_replica_n3] o.a.s.s.IndexSchema [testcollection_shard3_replica_n3] Schema name=test
   [junit4]   2> 673769 INFO  (parallelCoreAdminExecutor-807-thread-2-processing-n:127.0.0.1:46074_solr x:testcollection_shard2_replica_n2 da4723d5-29e8-4413-a7f0-19be46ddf7748869813676680554 CREATE) [n:127.0.0.1:46074_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 673820 WARN  (parallelCoreAdminExecutor-811-thread-1-processing-n:127.0.0.1:37652_solr x:testcollection_shard1_replica_n1 da4723d5-29e8-4413-a7f0-19be46ddf7748869813660738003 CREATE) [n:127.0.0.1:37652_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 673820 WARN  (parallelCoreAdminExecutor-811-thread-1-processing-n:127.0.0.1:37652_solr x:testcollection_shard1_replica_n1 da4723d5-29e8-4413-a7f0-19be46ddf7748869813660738003 CREATE) [n:127.0.0.1:37652_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 673836 INFO  (parallelCoreAdminExecutor-811-thread-1-processing-n:127.0.0.1:37652_solr x:testcollection_shard1_replica_n1 da4723d5-29e8-4413-a7f0-19be46ddf7748869813660738003 CREATE) [n:127.0.0.1:37652_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 673836 INFO  (parallelCoreAdminExecutor-811-thread-1-processing-n:127.0.0.1:37652_solr x:testcollection_shard1_replica_n1 da4723d5-29e8-4413-a7f0-19be46ddf7748869813660738003 CREATE) [n:127.0.0.1:37652_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n1' using configuration from collection testcollection, trusted=true
   [junit4]   2> 673837 INFO  (parallelCoreAdminExecutor-811-thread-1-processing-n:127.0.0.1:37652_solr x:testcollection_shard1_replica_n1 da4723d5-29e8-4413-a7f0-19be46ddf7748869813660738003 CREATE) [n:127.0.0.1:37652_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37652.solr.core.testcollection.shard1.replica_n1' (registry 'solr.core.testcollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3ef1f3
   [junit4]   2> 673837 INFO  (parallelCoreAdminExecutor-811-thread-1-processing-n:127.0.0.1:37652_solr x:testcollection_shard1_replica_n1 da4723d5-29e8-4413-a7f0-19be46ddf7748869813660738003 CREATE) [n:127.0.0.1:37652_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 673837 INFO  (parallelCoreAdminExecutor-811-thread-1-processing-n:127.0.0.1:37652_solr x:testcollection_shard1_replica_n1 da4723d5-29e8-4413-a7f0-19be46ddf7748869813660738003 CREATE) [n:127.0.0.1:37652_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [[testcollection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_9BB942761B40B0A2-001/tempDir-001/node2/testcollection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_9BB942761B40B0A2-001/tempDir-001/node2/./testcollection_shard1_replica_n1/data/]
   [junit4]   2> 673843 INFO  (parallelCoreAdminExecutor-807-thread-2-processing-n:127.0.0.1:46074_solr x:testcollection_shard2_replica_n2 da4723d5-29e8-4413-a7f0-19be46ddf7748869813676680554 CREATE) [n:127.0.0.1:46074_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n2] o.a.s.s.IndexSchema [testcollection_shard2_replica_n2] Schema name=test
   [junit4]   2> 674323 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : da4723d5-29e8-4413-a7f0-19be46ddf7748869813695690926
   [junit4]   2> 674323 INFO  (qtp447947068-2800) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=da4723d5-29e8-4413-a7f0-19be46ddf7748869813695690926&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 674408 INFO  (qtp447947068-2798) [n:127.0.0.1:35409_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=da4723d5-29e8-4413-a7f0-19be46ddf774&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 674409 INFO  (qtp447947068-2798) [n:127.0.0.1:35409_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=da4723d5-29e8-4413-a7f0-19be46ddf774&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 674863 WARN  (parallelCoreAdminExecutor-805-thread-1-processing-n:127.0.0.1:43619_solr x:testcollection_shard4_replica_n4 da4723d5-29e8-4413-a7f0-19be46ddf7748869813697424344 CREATE) [n:127.0.0.1:43619_solr c:testcollection s:shard4 r:core_node8 x:testcollection_shard4_replica_n4] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 674863 WARN  (parallelCoreAdminExecutor-805-thread-1-processing-n:127.0.0.1:43619_solr x:testcollection_shard4_replica_n4 da4723d5-29e8-4413-a7f0-19be46ddf7748869813697424344 CREATE) [n:127.0.0.1:43619_solr c:testcollection s:shard4 r:core_node8 x:testcollection_shard4_replica_n4] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 674883 INFO  (parallelCoreAdminExecutor-805-thread-1-processing-n:127.0.0.1:43619_solr x:testcollection_shard4_replica_n4 da4723d5-29e8-4413-a7f0-19be46ddf7748869813697424344 CREATE) [n:127.0.0.1:43619_solr c:testcollection s:shard4 r:core_node8 x:testcollection_shard4_replica_n4] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 674883 INFO  (parallelCoreAdminExecutor-805-thread-1-processing-n:127.0.0.1:43619_solr x:testcollection_shard4_replica_n4 da4723d5-29e8-4413-a7f0-19be46ddf7748869813697424344 CREATE) [n:127.0.0.1:43619_solr c:testcollection s:shard4 r:core_node8 x:testcollection_shard4_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard4_replica_n4' using configuration from collection testcollection, trusted=true
   [junit4]   2> 674883 INFO  (parallelCoreAdminExecutor-805-thread-1-processing-n:127.0.0.1:43619_solr x:testcollection_shard4_replica_n4 da4723d5-29e8-4413-a7f0-19be46ddf7748869813697424344 CREATE) [n:127.0.0.1:43619_solr c:testcollection s:shard4 r:core_node8 x:testcollection_shard4_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43619.solr.core.testcollection.shard4.replica_n4' (registry 'solr.core.testcollection.shard4.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3ef1f3
   [junit4]   2> 674883 INFO  (parallelCoreAdminExecutor-805-thread-1-processing-n:127.0.0.1:43619_solr x:testcollection_shard4_replica_n4 da4723d5-29e8-4413-a7f0-19be46ddf7748869813697424344 CREATE) [n:127.0.0.1:43619_solr c:testcollection s:shard4 r:core_node8 x:testcollection_shard4_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 674883 INFO  (parallelCoreAdminExecutor-805-thread-1-processing-n:127.0.0.1:43619_solr x:testcollection_shard4_replica_n4 da4723d5-29e8-4413-a7f0-19be46ddf7748869813697424344 CREATE) [n:127.0.0.1:43619_solr c:testcollection s:shard4 r:core_node8 x:testcollection_shard4_replica_n4] o.a.s.c.SolrCore [[testcollection_shard4_replica_n4] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_9BB942761B40B0A2-001/tempDir-001/node1/testcollection_shard4_replica_n4], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_9BB942761B40B0A2-001/tempDir-001/node1/./testcollection_shard4_replica_n4/data/]
   [junit4]   2> 674883 WARN  (parallelCoreAdminExecutor-808-thread-1-processing-n:127.0.0.1:35409_solr x:testcollection_shard3_replica_n3 da4723d5-29e8-4413-a7f0-19be46ddf7748869813695690926 CREATE) [n:127.0.0.1:35409_solr c:testcollection s:shard3 r:core_node7 x:testcollection_shard3_replica_n3] o.a.s.s.IndexSche

[...truncated too long message...]

ngs.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null721961926
     [copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null721961926
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null721961926

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null1571177919
     [copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null1571177919
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null1571177919

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 60ms :: artifacts dl 1ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 286 minutes 44 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2532 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2532/

2 tests failed.
FAILED:  org.apache.solr.cloud.TestCloudPivotFacet.test

Error Message:
{main(facet=true&facet.pivot={!stats%3Dst3}pivot_x_s,pivot_x_s,pivot_i1&facet.limit=3&facet.offset=6&facet.pivot.mincount=2&facet.missing=true&facet.sort=index&facet.overrequest.count=0&facet.overrequest.ratio=0),extra(rows=0&q=id:[*+TO+381]&stats=true&stats.field={!key%3Dsk1+tag%3Dst1,st2}pivot_dt&stats.field={!key%3Dsk2+tag%3Dst2,st3}pivot_i&stats.field={!key%3Dsk3+tag%3Dst3,st4}pivot_z_s&_test_min=2&_test_miss=true&_test_sort=index)} ==> pivot_x_s,pivot_x_s,pivot_i1: {params(rows=0),defaults({main({main(rows=0&q=id:[*+TO+381]&stats=true&stats.field={!key%3Dsk1+tag%3Dst1,st2}pivot_dt&stats.field={!key%3Dsk2+tag%3Dst2,st3}pivot_i&stats.field={!key%3Dsk3+tag%3Dst3,st4}pivot_z_s&_test_min=2&_test_miss=true&_test_sort=index),extra(fq={!term+f%3Dpivot_x_s}g)}),extra(fq={!term+f%3Dpivot_x_s}l)})} expected:<2> but was:<3>

Stack Trace:
java.lang.AssertionError: {main(facet=true&facet.pivot={!stats%3Dst3}pivot_x_s,pivot_x_s,pivot_i1&facet.limit=3&facet.offset=6&facet.pivot.mincount=2&facet.missing=true&facet.sort=index&facet.overrequest.count=0&facet.overrequest.ratio=0),extra(rows=0&q=id:[*+TO+381]&stats=true&stats.field={!key%3Dsk1+tag%3Dst1,st2}pivot_dt&stats.field={!key%3Dsk2+tag%3Dst2,st3}pivot_i&stats.field={!key%3Dsk3+tag%3Dst3,st4}pivot_z_s&_test_min=2&_test_miss=true&_test_sort=index)} ==> pivot_x_s,pivot_x_s,pivot_i1: {params(rows=0),defaults({main({main(rows=0&q=id:[*+TO+381]&stats=true&stats.field={!key%3Dsk1+tag%3Dst1,st2}pivot_dt&stats.field={!key%3Dsk2+tag%3Dst2,st3}pivot_i&stats.field={!key%3Dsk3+tag%3Dst3,st4}pivot_z_s&_test_min=2&_test_miss=true&_test_sort=index),extra(fq={!term+f%3Dpivot_x_s}g)}),extra(fq={!term+f%3Dpivot_x_s}l)})} expected:<2> but was:<3>
        at __randomizedtesting.SeedInfo.seed([D144A70E42EC6D6D:591098D4EC100095]:0)
        at org.apache.solr.cloud.TestCloudPivotFacet.assertPivotCountsAreCorrect(TestCloudPivotFacet.java:291)
        at org.apache.solr.cloud.TestCloudPivotFacet.test(TestCloudPivotFacet.java:238)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.AssertionError: pivot_x_s,pivot_x_s,pivot_i1: {params(rows=0),defaults({main({main(rows=0&q=id:[*+TO+381]&stats=true&stats.field={!key%3Dsk1+tag%3Dst1,st2}pivot_dt&stats.field={!key%3Dsk2+tag%3Dst2,st3}pivot_i&stats.field={!key%3Dsk3+tag%3Dst3,st4}pivot_z_s&_test_min=2&_test_miss=true&_test_sort=index),extra(fq={!term+f%3Dpivot_x_s}g)}),extra(fq={!term+f%3Dpivot_x_s}l)})} expected:<2> but was:<3>
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.apache.solr.cloud.TestCloudPivotFacet.assertNumFound(TestCloudPivotFacet.java:689)
        at org.apache.solr.cloud.TestCloudPivotFacet.assertPivotData(TestCloudPivotFacet.java:345)
        at org.apache.solr.cloud.TestCloudPivotFacet.assertPivotCountsAreCorrect(TestCloudPivotFacet.java:312)
        at org.apache.solr.cloud.TestCloudPivotFacet.assertPivotCountsAreCorrect(TestCloudPivotFacet.java:323)
        at org.apache.solr.cloud.TestCloudPivotFacet.assertPivotCountsAreCorrect(TestCloudPivotFacet.java:281)
        ... 42 more


FAILED:  org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest.testCreateNodeSet

Error Message:
Could not find collection : nodeset_collection

Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : nodeset_collection
        at __randomizedtesting.SeedInfo.seed([D144A70E42EC6D6D:8AE770EE2B6D49E5]:0)
        at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:118)
        at org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:256)
        at org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest.testCreateNodeSet(CollectionsAPIDistributedZkTest.java:408)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13236 lines...]
   [junit4] Suite: org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest
   [junit4]   2> 1085314 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D144A70E42EC6D6D]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIDistributedZkTest_D144A70E42EC6D6D-001/init-core-data-001
   [junit4]   2> 1085315 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D144A70E42EC6D6D]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1085316 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D144A70E42EC6D6D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 1085317 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D144A70E42EC6D6D]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIDistributedZkTest_D144A70E42EC6D6D-001/tempDir-001
   [junit4]   2> 1085317 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D144A70E42EC6D6D]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1085330 INFO  (Thread-3509) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1085330 INFO  (Thread-3509) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1085350 ERROR (Thread-3509) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1085494 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D144A70E42EC6D6D]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:34941
   [junit4]   2> 1085563 INFO  (zkConnectionManagerCallback-1505-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1085596 INFO  (jetty-launcher-1502-thread-2) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 1085600 INFO  (jetty-launcher-1502-thread-3) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 1085601 INFO  (jetty-launcher-1502-thread-4) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 1085615 INFO  (jetty-launcher-1502-thread-1) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 1085674 INFO  (jetty-launcher-1502-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1085674 INFO  (jetty-launcher-1502-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1085674 INFO  (jetty-launcher-1502-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1085679 INFO  (jetty-launcher-1502-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@13aea73d{/solr,null,AVAILABLE}
   [junit4]   2> 1085681 INFO  (jetty-launcher-1502-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4aa025a4{HTTP/1.1,[http/1.1]}{127.0.0.1:44117}
   [junit4]   2> 1085681 INFO  (jetty-launcher-1502-thread-2) [    ] o.e.j.s.Server Started @1085740ms
   [junit4]   2> 1085681 INFO  (jetty-launcher-1502-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44117}
   [junit4]   2> 1085682 ERROR (jetty-launcher-1502-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1085682 INFO  (jetty-launcher-1502-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1085682 INFO  (jetty-launcher-1502-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1085682 INFO  (jetty-launcher-1502-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1085682 INFO  (jetty-launcher-1502-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1085682 INFO  (jetty-launcher-1502-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-17T17:54:44.042Z
   [junit4]   2> 1085746 INFO  (jetty-launcher-1502-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1085746 INFO  (jetty-launcher-1502-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1085746 INFO  (jetty-launcher-1502-thread-3) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1085748 INFO  (jetty-launcher-1502-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@c151975{/solr,null,AVAILABLE}
   [junit4]   2> 1085748 INFO  (jetty-launcher-1502-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2a145771{HTTP/1.1,[http/1.1]}{127.0.0.1:52047}
   [junit4]   2> 1085748 INFO  (jetty-launcher-1502-thread-3) [    ] o.e.j.s.Server Started @1085807ms
   [junit4]   2> 1085748 INFO  (jetty-launcher-1502-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52047}
   [junit4]   2> 1085748 ERROR (jetty-launcher-1502-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1085749 INFO  (jetty-launcher-1502-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1085749 INFO  (jetty-launcher-1502-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1085749 INFO  (jetty-launcher-1502-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1085749 INFO  (jetty-launcher-1502-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1085749 INFO  (jetty-launcher-1502-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-17T17:54:44.109Z
   [junit4]   2> 1085805 INFO  (jetty-launcher-1502-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1085805 INFO  (jetty-launcher-1502-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1085805 INFO  (jetty-launcher-1502-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1085807 INFO  (jetty-launcher-1502-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@74f81ffc{/solr,null,AVAILABLE}
   [junit4]   2> 1085807 INFO  (jetty-launcher-1502-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2c203ec2{HTTP/1.1,[http/1.1]}{127.0.0.1:57265}
   [junit4]   2> 1085807 INFO  (jetty-launcher-1502-thread-1) [    ] o.e.j.s.Server Started @1085867ms
   [junit4]   2> 1085807 INFO  (jetty-launcher-1502-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57265}
   [junit4]   2> 1085808 ERROR (jetty-launcher-1502-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1085808 INFO  (jetty-launcher-1502-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1085808 INFO  (jetty-launcher-1502-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1085808 INFO  (jetty-launcher-1502-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1085808 INFO  (jetty-launcher-1502-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1085808 INFO  (jetty-launcher-1502-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-17T17:54:44.168Z
   [junit4]   2> 1085857 INFO  (jetty-launcher-1502-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1085857 INFO  (jetty-launcher-1502-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1085857 INFO  (jetty-launcher-1502-thread-4) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1085857 INFO  (jetty-launcher-1502-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@30bcbd42{/solr,null,AVAILABLE}
   [junit4]   2> 1085871 INFO  (zkConnectionManagerCallback-1509-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1085872 INFO  (jetty-launcher-1502-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1085875 INFO  (jetty-launcher-1502-thread-3) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1085875 INFO  (jetty-launcher-1502-thread-3) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1085878 INFO  (jetty-launcher-1502-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@17dfacce{HTTP/1.1,[http/1.1]}{127.0.0.1:38031}
   [junit4]   2> 1085878 INFO  (jetty-launcher-1502-thread-4) [    ] o.e.j.s.Server Started @1085937ms
   [junit4]   2> 1085878 INFO  (jetty-launcher-1502-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38031}
   [junit4]   2> 1085878 ERROR (jetty-launcher-1502-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1085878 INFO  (jetty-launcher-1502-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1085878 INFO  (jetty-launcher-1502-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1085878 INFO  (jetty-launcher-1502-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1085878 INFO  (jetty-launcher-1502-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1085878 INFO  (jetty-launcher-1502-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-17T17:54:44.238Z
   [junit4]   2> 1085880 INFO  (zkConnectionManagerCallback-1507-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1085880 INFO  (jetty-launcher-1502-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1085884 INFO  (jetty-launcher-1502-thread-2) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1085884 INFO  (jetty-launcher-1502-thread-2) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1085908 INFO  (jetty-launcher-1502-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34941/solr
   [junit4]   2> 1085918 INFO  (zkConnectionManagerCallback-1511-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1085918 INFO  (jetty-launcher-1502-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1085922 INFO  (jetty-launcher-1502-thread-1) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1085922 INFO  (jetty-launcher-1502-thread-1) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1085940 INFO  (zkConnectionManagerCallback-1517-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1085940 INFO  (jetty-launcher-1502-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34941/solr
   [junit4]   2> 1085941 INFO  (zkConnectionManagerCallback-1513-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1085942 INFO  (jetty-launcher-1502-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1086062 INFO  (jetty-launcher-1502-thread-4) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1086062 INFO  (jetty-launcher-1502-thread-4) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1086084 INFO  (jetty-launcher-1502-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34941/solr
   [junit4]   2> 1086098 INFO  (zkConnectionManagerCallback-1523-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1086106 INFO  (zkConnectionManagerCallback-1527-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1086136 INFO  (zkConnectionManagerCallback-1521-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1086138 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1063f496e440007, likely client has closed socket
   [junit4]   2> 1086212 INFO  (jetty-launcher-1502-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34941/solr
   [junit4]   2> 1086214 INFO  (jetty-launcher-1502-thread-3) [n:127.0.0.1:52047_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1086215 INFO  (jetty-launcher-1502-thread-3) [n:127.0.0.1:52047_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52047_solr
   [junit4]   2> 1086216 INFO  (jetty-launcher-1502-thread-3) [n:127.0.0.1:52047_solr    ] o.a.s.c.Overseer Overseer (id=73816028513304582-127.0.0.1:52047_solr-n_0000000000) starting
   [junit4]   2> 1086223 INFO  (zkConnectionManagerCallback-1531-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1086226 INFO  (zkConnectionManagerCallback-1533-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1086241 INFO  (jetty-launcher-1502-thread-2) [n:127.0.0.1:44117_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1086242 INFO  (jetty-launcher-1502-thread-2) [n:127.0.0.1:44117_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44117_solr
   [junit4]   2> 1086255 INFO  (zkConnectionManagerCallback-1537-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1086300 INFO  (jetty-launcher-1502-thread-1) [n:127.0.0.1:57265_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1086303 INFO  (jetty-launcher-1502-thread-1) [n:127.0.0.1:57265_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1086303 INFO  (jetty-launcher-1502-thread-1) [n:127.0.0.1:57265_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57265_solr
   [junit4]   2> 1086339 INFO  (zkCallback-1530-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1086339 INFO  (zkCallback-1522-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1086359 INFO  (zkCallback-1530-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1086367 INFO  (zkConnectionManagerCallback-1546-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1086367 INFO  (zkCallback-1532-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1086369 INFO  (jetty-launcher-1502-thread-3) [n:127.0.0.1:52047_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1086370 INFO  (jetty-launcher-1502-thread-3) [n:127.0.0.1:52047_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34941/solr ready
   [junit4]   2> 1086392 INFO  (jetty-launcher-1502-thread-3) [n:127.0.0.1:52047_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1086392 INFO  (jetty-launcher-1502-thread-3) [n:127.0.0.1:52047_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52047_solr
   [junit4]   2> 1086399 INFO  (zkCallback-1522-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1086597 INFO  (zkCallback-1532-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1086598 INFO  (zkCallback-1530-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1086606 INFO  (zkConnectionManagerCallback-1548-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1086613 INFO  (jetty-launcher-1502-thread-4) [n:127.0.0.1:38031_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1086616 INFO  (jetty-launcher-1502-thread-4) [n:127.0.0.1:38031_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1086711 INFO  (zkCallback-1545-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1086713 INFO  (jetty-launcher-1502-thread-4) [n:127.0.0.1:38031_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1086713 INFO  (jetty-launcher-1502-thread-4) [n:127.0.0.1:38031_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38031_solr
   [junit4]   2> 1086714 INFO  (zkCallback-1522-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1086714 INFO  (zkCallback-1532-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1086715 INFO  (zkCallback-1530-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1086804 INFO  (zkCallback-1545-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1086886 INFO  (zkCallback-1547-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1086975 INFO  (jetty-launcher-1502-thread-1) [n:127.0.0.1:57265_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43f42aa2
   [junit4]   2> 1087010 INFO  (jetty-launcher-1502-thread-1) [n:127.0.0.1:57265_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43f42aa2
   [junit4]   2> 1087011 INFO  (jetty-launcher-1502-thread-1) [n:127.0.0.1:57265_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43f42aa2
   [junit4]   2> 1087024 INFO  (jetty-launcher-1502-thread-1) [n:127.0.0.1:57265_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIDistributedZkTest_D144A70E42EC6D6D-001/tempDir-001/node1
   [junit4]   2> 1087060 INFO  (jetty-launcher-1502-thread-3) [n:127.0.0.1:52047_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43f42aa2
   [junit4]   2> 1087090 INFO  (zkConnectionManagerCallback-1555-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1087095 INFO  (jetty-launcher-1502-thread-1) [n:127.0.0.1:57265_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1087100 INFO  (jetty-launcher-1502-thread-3) [n:127.0.0.1:52047_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43f42aa2
   [junit4]   2> 1087101 INFO  (jetty-launcher-1502-thread-3) [n:127.0.0.1:52047_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43f42aa2
   [junit4]   2> 1087102 INFO  (jetty-launcher-1502-thread-3) [n:127.0.0.1:52047_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIDistributedZkTest_D144A70E42EC6D6D-001/tempDir-001/node3
   [junit4]   2> 1087103 INFO  (jetty-launcher-1502-thread-1) [n:127.0.0.1:57265_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34941/solr ready
   [junit4]   2> 1087288 INFO  (jetty-launcher-1502-thread-4) [n:127.0.0.1:38031_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43f42aa2
   [junit4]   2> 1087308 INFO  (jetty-launcher-1502-thread-4) [n:127.0.0.1:38031_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43f42aa2
   [junit4]   2> 1087308 INFO  (jetty-launcher-1502-thread-4) [n:127.0.0.1:38031_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43f42aa2
   [junit4]   2> 1087310 INFO  (jetty-launcher-1502-thread-4) [n:127.0.0.1:38031_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIDistributedZkTest_D144A70E42EC6D6D-001/tempDir-001/node4
   [junit4]   2> 1087379 INFO  (zkConnectionManagerCallback-1560-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1087380 INFO  (jetty-launcher-1502-thread-4) [n:127.0.0.1:38031_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1087381 INFO  (jetty-launcher-1502-thread-4) [n:127.0.0.1:38031_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34941/solr ready
   [junit4]   2> 1087409 INFO  (jetty-launcher-1502-thread-2) [n:127.0.0.1:44117_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43f42aa2
   [junit4]   2> 1087431 INFO  (jetty-launcher-1502-thread-2) [n:127.0.0.1:44117_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43f42aa2
   [junit4]   2> 1087431 INFO  (jetty-launcher-1502-thread-2) [n:127.0.0.1:44117_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43f42aa2
   [junit4]   2> 1087433 INFO  (jetty-launcher-1502-thread-2) [n:127.0.0.1:44117_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIDistributedZkTest_D144A70E42EC6D6D-001/tempDir-001/node2
   [junit4]   2> 1087471 INFO  (zkConnectionManagerCallback-1565-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1087472 INFO  (jetty-launcher-1502-thread-2) [n:127.0.0.1:44117_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1087474 INFO  (jetty-launcher-1502-thread-2) [n:127.0.0.1:44117_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34941/solr ready
   [junit4]   2> 1087883 INFO  (zkConnectionManagerCallback-1567-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1087919 INFO  (zkConnectionManagerCallback-1572-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1087920 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D144A70E42EC6D6D]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1087921 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[D144A70E42EC6D6D]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34941/solr ready
   [junit4]   2> 1088063 INFO  (TEST-CollectionsAPIDistributedZkTest.testZeroNumShards-seed#[D144A70E42EC6D6D]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testZeroNumShards
   [junit4]   2> 1088065 INFO  (TEST-CollectionsAPIDistributedZkTest.testZeroNumShards-seed#[D144A70E42EC6D6D]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1088072 INFO  (qtp1372912206-8484) [n:127.0.0.1:38031_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=10&collection.configName=conf&name=acollection&action=CREATE&numShards=0&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1088087 INFO  (OverseerThreadFactory-2498-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection acollection
   [junit4]   2> 1088089 ERROR (OverseerThreadFactory-2498-thread-1) [    ] o.a.s.c.a.c.OverseerCollectionMessageHandler Collection: acollection operation: create failed:org.apache.solr.common.SolrException: numShards must be > 0
   [junit4]   2> at org.apache.solr.cloud.api.collections.CreateCollectionCmd.buildReplicaPositions(CreateCollectionCmd.java:287)
   [junit4]   2> at org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:123)
   [junit4]   2> at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:257)
   [junit4]   2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:469)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>
   [junit4]   2> 1088093 INFO  (qtp1372912206-8484) [n:127.0.0.1:38031_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=10&collection.configName=conf&name=acollection&action=CREATE&numShards=0&wt=javabin&version=2} status=400 QTime=20
   [junit4]   2> 1088093 INFO  (TEST-CollectionsAPIDistributedZkTest.testZeroNumShards-seed#[D144A70E42EC6D6D]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testZeroNumShards
   [junit4]   2> 1088272 INFO  (TEST-CollectionsAPIDistributedZkTest.testCreateNodeSet-seed#[D144A70E42EC6D6D]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCreateNodeSet
   [junit4]   2> 1088276 INFO  (TEST-CollectionsAPIDistributedZkTest.testCreateNodeSet-seed#[D144A70E42EC6D6D]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1088278 INFO  (qtp1372912206-8480) [n:127.0.0.1:38031_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=nodeset_collection&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=http://127.0.0.1:57265/solr,http://127.0.0.1:38031/solr&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1088298 INFO  (OverseerThreadFactory-2498-thread-2) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection nodeset_collection
   [junit4]   2> 1088298 INFO  (OverseerCollectionConfigSetProcessor-73816028513304582-127.0.0.1:52047_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1088302 WARN  (OverseerThreadFactory-2498-thread-2) [    ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (nodeset_collection) without cores.
   [junit4]   2> 1088491 INFO  (qtp1372912206-8480) [n:127.0.0.1:38031_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1088492 INFO  (qtp1372912206-8480) [n:127.0.0.1:38031_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=nodeset_collection&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=http://127.0.0.1:57265/solr,http://127.0.0.1:38031/solr&wt=javabin&version=2} status=0 QTime=214
   [junit4]   2> 1088493 INFO  (TEST-CollectionsAPIDistributedZkTest.testCreateNodeSet-seed#[D144A70E42EC6D6D]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testCreateNodeSet
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=testCreateNodeSet -Dtests.seed=D144A70E42EC6D6D -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=hu-HU -Dtests.timezone=America/Araguaina -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.40s J1 | CollectionsAPIDistributedZkTest.testCreateNodeSet <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: Could not find collection : nodeset_collection
   [junit4]    > at __randomizedtesting.SeedInfo.seed([D144A70E42EC6D6D:8AE770EE2B6D49E5]:0)
   [junit4]    > at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:118)
   [junit4]    > at org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:256)
   [junit4]    > at org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest.testCreateNodeSet(CollectionsAPIDistributedZkTest.java:408)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1088669 INFO  (TEST-CollectionsAPIDistributedZkTest.testMissingNumShards-seed#[D144A70E42EC6D6D]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMissingNumShards
   [junit4]   2> 1088670 INFO  (TEST-CollectionsAPIDistributedZkTest.testMissingNumShards-seed#[D144A70E42EC6D6D]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1088675 INFO  (qtp1372912206-8483) [n:127.0.0.1:38031_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=nodeset_collection&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1088708 INFO  (OverseerCollectionConfigSetProcessor-73816028513304582-127.0.0.1:52047_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1088711 INFO  (OverseerThreadFactory-2498-thread-3) [    ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd=action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true, asyncId=null
   [junit4]   2> 1089529 INFO  (qtp1372912206-8483) [n:127.0.0.1:38031_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=nodeset_collection&action=DELETE&wt=javabin&version=2} status=0 QTime=854
   [junit4]   2> 1089530 INFO  (qtp1372912206-8485) [n:127.0.0.1:38031_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=10&collection.configName=conf&name=acollection&action=CREATE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1089564 INFO  (OverseerCollectionConfigSetProcessor-73816028513304582-127.0.0.1:52047_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1089565 INFO  (OverseerThreadFactory-2498-thread-4) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection acollection
   [junit4]   2> 1089566 ERROR (OverseerThreadFactory-2498-thread-4) [    ] o.a.s.c.a.c.OverseerCollectionMessageHandler Collection: acollection operation: create failed:org.apache.solr.common.SolrException: numShards is a required param (when using CompositeId router).
   [junit4]   2> at org.apache.solr.cloud.api.collections.CreateCollectionCmd.buildReplicaPositions(CreateCollectionCmd.java:284)
   [junit4]   2> at org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:123)
   [junit4]   2> at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:257)
   [junit4]   2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:469)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>
   [junit4]   2> 1089571 INFO  (qtp1372912206-8485) [n:127.0.0.1:38031_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=10&collection.configName=conf&name=acollection&action=CREATE&wt=javabin&version=2} status=400 QTime=40
   [junit4]   2> 1089572 INFO  (TEST-CollectionsAPIDistributedZkTest.testMissingNumShards-seed#[D144A70E42EC6D6D]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testMissingNumShards
   [junit4]   2> 1089627 INFO  (TEST-CollectionsAPIDistributedZkTest.testCollectionReload-seed#[D144A70E42EC6D6D]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCollectionReload
   [junit4]   2> 1089628 INFO  (TEST-CollectionsAPIDistributedZkTest.testCollectionReload-seed#[D144A70E42EC6D6D]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1089630 INFO  (qtp1372912206-8483) [n:127.0.0.1:38031_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=reloaded_collection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1089639 INFO  (OverseerCollectionConfigSetProcessor-73816028513304582-127.0.0.1:52047_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1089640 INFO  (OverseerThreadFactory-2498-thread-5) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection reloaded_collection
   [junit4]   2> 1089854 INFO  (OverseerStateUpdate-73816028513304582-127.0.0.1:52047_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"reloaded_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"reloaded_collection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:52047/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1089858 INFO  (OverseerStateUpdate-73816028513304582-127.0.0.1:52047_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"reloaded_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"reloaded_collection_shard1_replica_n3",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:57265/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1089861 INFO  (OverseerStateUpdate-73816028513304582-127.0.0.1:52047_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"reloaded_collection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"reloaded_collection_shard2_replica_n5",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:38031/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1089865 INFO  (OverseerStateUpdate-73816028513304582-127.0.0.1:52047_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"reloaded_collection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"reloaded_collection_shard2_replica_n7",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:44117/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1090109 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr    x:reloaded_collection_shard2_replica_n5] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&newCollection=true&name=reloaded_collection_shard2_replica_n5&action=CREATE&numShards=2&collection=reloaded_collection&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1090117 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr    x:reloaded_collection_shard1_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=reloaded_collection_shard1_replica_n3&action=CREATE&numShards=2&collection=reloaded_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1090120 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr    x:reloaded_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=reloaded_collection_shard1_replica_n1&action=CREATE&numShards=2&collection=reloaded_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1090132 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr    x:reloaded_collection_shard2_replica_n7] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=reloaded_collection_shard2_replica_n7&action=CREATE&numShards=2&collection=reloaded_collection&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1090132 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr    x:reloaded_collection_shard2_replica_n7] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1091155 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1091173 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1091176 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1091187 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1091190 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.s.IndexSchema [reloaded_collection_shard1_replica_n3] Schema name=minimal
   [junit4]   2> 1091190 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.s.IndexSchema [reloaded_collection_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 1091195 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.s.IndexSchema [reloaded_collection_shard2_replica_n5] Schema name=minimal
   [junit4]   2> 1091199 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1091199 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'reloaded_collection_shard1_replica_n3' using configuration from collection reloaded_collection, trusted=true
   [junit4]   2> 1091199 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.reloaded_collection.shard1.replica_n3' (registry 'solr.core.reloaded_collection.shard1.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43f42aa2
   [junit4]   2> 1091199 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1091200 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.c.SolrCore [[reloaded_collection_shard1_replica_n3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIDistributedZkTest_D144A70E42EC6D6D-001/tempDir-001/node1/reloaded_collection_shard1_replica_n3], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIDistributedZkTest_D144A70E42EC6D6D-001/tempDir-001/node1/reloaded_collection_shard1_replica_n3/data/]
   [junit4]   2> 1091250 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1091250 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'reloaded_collection_shard1_replica_n1' using configuration from collection reloaded_collection, trusted=true
   [junit4]   2> 1091251 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.reloaded_collection.shard1.replica_n1' (registry 'solr.core.reloaded_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43f42aa2
   [junit4]   2> 1091251 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1091251 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.c.SolrCore [[reloaded_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIDistributedZkTest_D144A70E42EC6D6D-001/tempDir-001/node3/reloaded_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIDistributedZkTest_D144A70E42EC6D6D-001/tempDir-001/node3/reloaded_collection_shard1_replica_n1/data/]
   [junit4]   2> 1091255 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.s.IndexSchema [reloaded_collection_shard2_replica_n7] Schema name=minimal
   [junit4]   2> 1091258 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1091258 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.c.CoreContainer Creating SolrCore 'reloaded_collection_shard2_replica_n7' using configuration from collection reloaded_collection, trusted=true
   [junit4]   2> 1091258 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1091258 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'reloaded_collection_shard2_replica_n5' using configuration from collection reloaded_collection, trusted=true
   [junit4]   2> 1091259 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.reloaded_collection.shard2.replica_n5' (registry 'solr.core.reloaded_collection.shard2.replica_n5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43f42aa2
   [junit4]   2> 1091259 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1091259 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.c.SolrCore [[reloaded_collection_shard2_replica_n5] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIDistributedZkTest_D144A70E42EC6D6D-001/tempDir-001/node4/reloaded_collection_shard2_replica_n5], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIDistributedZkTest_D144A70E42EC6D6D-001/tempDir-001/node4/reloaded_collection_shard2_replica_n5/data/]
   [junit4]   2> 1091260 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.reloaded_collection.shard2.replica_n7' (registry 'solr.core.reloaded_collection.shard2.replica_n7') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@43f42aa2
   [junit4]   2> 1091260 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1091260 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.c.SolrCore [[reloaded_collection_shard2_replica_n7] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIDistributedZkTest_D144A70E42EC6D6D-001/tempDir-001/node2/reloaded_collection_shard2_replica_n7], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIDistributedZkTest_D144A70E42EC6D6D-001/tempDir-001/node2/reloaded_collection_shard2_replica_n7/data/]
   [junit4]   2> 1091664 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1091664 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1091665 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1091665 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1091667 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1091667 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1091668 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1091669 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1091670 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@64807869[reloaded_collection_shard1_replica_n1] main]
   [junit4]   2> 1091672 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1091673 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1091674 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1091674 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.s.SolrIndexSearcher Opening [Searcher@548d2993[reloaded_collection_shard2_replica_n5] main]
   [junit4]   2> 1091675 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1091675 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1091676 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1091676 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1091679 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1091679 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1091680 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1091681 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@62c8ea77[reloaded_collection_shard1_replica_n3] main]
   [junit4]   2> 1091681 INFO  (searcherExecutor-2510-thread-1-processing-n:127.0.0.1:38031_solr x:reloaded_collection_shard2_replica_n5 c:reloaded_collection s:shard2 r:core_node6) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.c.SolrCore [reloaded_collection_shard2_replica_n5] Registered new searcher Searcher@548d2993[reloaded_collection_shard2_replica_n5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1091682 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1091683 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1091683 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1091684 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600734825067577344
   [junit4]   2> 1091686 INFO  (searcherExecutor-2507-thread-1-processing-n:127.0.0.1:57265_solr x:reloaded_collection_shard1_replica_n3 c:reloaded_collection s:shard1 r:core_node4) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.c.SolrCore [reloaded_collection_shard1_replica_n3] Registered new searcher Searcher@62c8ea77[reloaded_collection_shard1_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1091689 INFO  (searcherExecutor-2508-thread-1-processing-n:127.0.0.1:52047_solr x:reloaded_collection_shard1_replica_n1 c:reloaded_collection s:shard1 r:core_node2) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.c.SolrCore [reloaded_collection_shard1_replica_n1] Registered new searcher Searcher@64807869[reloaded_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1091693 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/reloaded_collection/terms/shard1 to Terms{values={core_node4=0}, version=0}
   [junit4]   2> 1091698 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 1091708 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600734825092743168
   [junit4]   2> 1091710 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600734825094840320
   [junit4]   2> 1091731 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/reloaded_collection/terms/shard1 to Terms{values={core_node2=0, core_node4=0}, version=1}
   [junit4]   2> 1091733 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.c.ZkShardTerms Successful update of terms at /collections/reloaded_collection/terms/shard2 to Terms{values={core_node6=0}, version=0}
   [junit4]   2> 1091738 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 1091779 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1091779 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1091781 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1091781 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1091783 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.s.SolrIndexSearcher Opening [Searcher@3d2f81e9[reloaded_collection_shard2_replica_n7] main]
   [junit4]   2> 1091785 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1091785 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1091786 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1091787 INFO  (searcherExecutor-2509-thread-1-processing-n:127.0.0.1:44117_solr x:reloaded_collection_shard2_replica_n7 c:reloaded_collection s:shard2 r:core_node8) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.c.SolrCore [reloaded_collection_shard2_replica_n7] Registered new searcher Searcher@3d2f81e9[reloaded_collection_shard2_replica_n7] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1091788 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600734825176629248
   [junit4]   2> 1091794 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.c.ZkShardTerms Successful update of terms at /collections/reloaded_collection/terms/shard2 to Terms{values={core_node6=0, core_node8=0}, version=1}
   [junit4]   2> 1092200 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1092200 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1092200 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:57265/solr/reloaded_collection_shard1_replica_n3/
   [junit4]   2> 1092201 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=reloaded_collection_shard1_replica_n3 url=http://127.0.0.1:57265/solr START replicas=[http://127.0.0.1:52047/solr/reloaded_collection_shard1_replica_n1/] nUpdates=100
   [junit4]   2> 1092203 INFO  (qtp1301533120-8459) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.c.S.Request [reloaded_collection_shard1_replica_n1]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1092239 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1092240 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1092240 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:38031/solr/reloaded_collection_shard2_replica_n5/
   [junit4]   2> 1092240 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.u.PeerSync PeerSync: core=reloaded_collection_shard2_replica_n5 url=http://127.0.0.1:38031/solr START replicas=[http://127.0.0.1:44117/solr/reloaded_collection_shard2_replica_n7/] nUpdates=100
   [junit4]   2> 1092243 INFO  (qtp1381211665-8448) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.c.S.Request [reloaded_collection_shard2_replica_n7]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1092501 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=reloaded_collection_shard1_replica_n3 url=http://127.0.0.1:57265/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 1092502 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 1092502 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 1092502 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1092506 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:57265/solr/reloaded_collection_shard1_replica_n3/ shard1
   [junit4]   2> 1092541 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.u.PeerSync PeerSync: core=reloaded_collection_shard2_replica_n5 url=http://127.0.0.1:38031/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 1092541 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 1092541 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 1092541 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1092544 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:38031/solr/reloaded_collection_shard2_replica_n5/ shard2
   [junit4]   2> 1092650 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1092653 INFO  (qtp1372912206-8479) [n:127.0.0.1:38031_solr c:reloaded_collection s:shard2 r:core_node6 x:reloaded_collection_shard2_replica_n5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&newCollection=true&name=reloaded_collection_shard2_replica_n5&action=CREATE&numShards=2&collection=reloaded_collection&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2546
   [junit4]   2> 1092659 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1092662 INFO  (qtp1537997838-8475) [n:127.0.0.1:57265_solr c:reloaded_collection s:shard1 r:core_node4 x:reloaded_collection_shard1_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=reloaded_collection_shard1_replica_n3&action=CREATE&numShards=2&collection=reloaded_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2545
   [junit4]   2> 1092739 INFO  (qtp1301533120-8464) [n:127.0.0.1:52047_solr c:reloaded_collection s:shard1 r:core_node2 x:reloaded_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=reloaded_collection_shard1_replica_n1&action=CREATE&numShards=2&collection=reloaded_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2632
   [junit4]   2> 1092801 INFO  (qtp1381211665-8453) [n:127.0.0.1:44117_solr c:reloaded_collection s:shard2 r:core_node8 x:reloaded_collection_shard2_replica_n7] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=reloaded_collection_shard2_replica_n7&action=CREATE&numShards=2&collection=reloaded_collection&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2669
   [junit4]   2> 1092809 INFO  (qtp1372912206-8483) [n:127.0.0.1:38031_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1092906 INFO  (zkCallback-1532-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/reloaded_collection/state.json] for collection [reloaded_collection] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1092906 INFO  (zkCallback-1530-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/reloaded_collection/state.json] for collection [reloaded_collection] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1092906 INFO  (zkCallback-1547-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/reloaded_collection/state.json] for collection [reloaded_collection] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1092907 INFO  (zkCallback-1522-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/reloaded_collection/state.json] for collection [reloaded_collection] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1093642 INFO  (OverseerCollectionConfigSetProcessor-73816028513304582-127.0.0.1:52047_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1093809 INFO  (qtp1372912206-8483) [n:127.0.0.1:38031_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&name=reloaded_collection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=4178
   [junit4]   2> 1093813 INFO  (qtp1301533120-8460) [n:127.0.0.1:52047_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={action=STATUS&indexInfo=false&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1093823 INFO  (qtp1537997838-8470) [n:127.0.0.1:57265_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={action=STATUS&indexInfo=false&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1093828 INFO  (qtp1372912206-8485) [n:127.0.0.1:38031_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={action=STATUS&indexInfo=false&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1093834 INFO  (qtp1381211665-8449) [n:127.0.0.1:44117_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={action=STATUS&indexInfo=false&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 1093835 INFO  (qtp1372912206-8485) [n:127.0.0.1:38031_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :reload with params async=0742e45f-f136-4c1c-b25e-56309b8b3150&name=reloaded_collection&action=RELOAD&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1093844 INFO  (OverseerThreadFactory-2498-thread-5) [    ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd=action=RELOAD, asyncId=0742e45f-f136-4c1c-b25e-56309b8b3150
   [junit4]   2> 1093845 INFO  (qtp1372912206-8485) [n:127.0.0.1:38031_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=0742e45f-f136-4c1c-b25e-56309b8b3150&name=reloaded_collection&action=RELOAD&wt=javabin&version=2} status=0 QTime=10
   [junit4]   2> 1093849 INFO  (qtp1537997838-8471) [n:127.0.0.1:57265_solr    x:reloaded_collection_shard1_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={core=reloaded_collection_shard1_replica_n3&async=0742e45f-f136-4c1c-b25e-56309b8b315026831589245501904&qt=/admin/cores&action=RELOAD&wt=javabin&version=2} status=0 QTime=4
   [junit4]   2> 1093862 INFO  (parallelCoreAdminExecutor-2499-thread-1-processing-n:127.0.0.1:57265_solr x:reloaded_collection_shard1_replica_n3 0742e45f-f136-4c1c-b25e-56309b8b315026831589245501904 RELOAD) [n:127.0.0.1:57265_solr    x:reloaded_collection_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1093864 INFO  (qtp1301533120-8459) [n:127.0.0.1:52047_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={action=STATUS&indexInfo=false&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1093865 INFO  (qtp1372912206-8486) [n:127.0.0.1:38031_solr    x:reloaded_collection_shard2_replica_n5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={core=reloaded_collection_shard2_replica_n5&async=0742e45f-f136-4c1c-b25e-56309b8b315026831589245549864&qt=/admin/cores&action=RELOAD&wt=javabin&version=2} status=0 QTime=7
   [junit4]   2> 1093872 INFO  (qtp1301533120-8460) [n:127.0.0.1:52047_solr    x:reloaded_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={core=reloaded_collection_shard1_replica_n1&async=0742e45f-f136-4c1c-b25e-56309b8b315026831589245425078&qt=/admin/cores&action=RELOAD&wt=javabin&version=2} status=0 QTime=28
   [junit4]   2> 1093885 INFO  (qtp1381211665-8452) [n:127.0.0.1:44117_solr    x:reloaded_collection_shard2_replica_n7] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={core=reloaded_collection_shard2_replica_n7&async=0742e45f-f136-4c1c-b25e-56309b8b315026831589258516231&qt=/admin/cores&action=RELOAD&wt=javabin&version=2} status=0 QTime=22
   [junit4]   2> 1093887 INFO  (qtp1372912206-8485) [n:127.0.0.1:38031_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 0742e45f-f136-4c1c-b25e-56309b8b315026831589245549864
   [junit4]   2> 1093889 INFO  (qtp1537997838-8471) [n:127.0.0.1:57265_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={action=STATUS&indexInfo=false&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1093893 INFO  (parallelCoreAdminExecutor-2499-thread-1-processing-n:127.0.0.1:57265_solr x:reloaded_collection_shard1_replica_n3 0742e45f-f136-4c1c-b25e-56309b8b315026831589245501904 RELOAD) [n:127.0.0.1:57265_solr    x:reloaded_collection_shard1_replica_n3] o.a.s.s.IndexSchema [reloaded_collection_shard1_replica_n3] Schema name=minimal
   [junit4]   2> 1093896 INFO  (parallelCoreAdminExecutor-2499-thread-1-processing-n:127.0.0.1:57265_solr x:reloaded_collection_shard1_replica_n3 0742e45f-f136-4c1c-b25e-56309b8b315026831589245501904 RELOAD) [n:127.0.0.1:57265_solr    x:reloaded_collection_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1093896 INFO  (parallelCoreAdminExecutor-2499-thread-1-processing-n:127.0.0.1:57265_solr x:reloaded_collection_shard1_replica_n3 0742e45f-f136-4c1c-b25e-56309b8b315026831589245501904 RELOAD) [n:127.0.0.1:57265_solr    x:reloaded_collection_shard1_replica_n3] o.a.s.c.CoreContainer Reloading SolrCore 'reloaded_collection_shard1_replica_n3' using configuration from collection reloaded_collection
   [junit4]   2> 1093898 INFO  (qtp1372912206-8486) [n:127.0.0.1:38031_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={action=STATUS&indexInfo=false&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1093899 INFO  (qtp1372912206-8485) [n:127.0.0.1:38031_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=0742e45f-f136-4c1c-b25e-56309b8b315026831589245549864&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=12
   [junit4]   2> 1093954 INFO  (parallelCoreAdminExecutor-2503-thread-1-processing-n:127.0.0.1:38031_solr x:reloaded_collection_shard2_replica_n5 0742e45f-f136-4c1c-b25e-56309b8b315026831589245549864 RELOAD) [n:127.0.0.1:38031_solr    x:reloaded_collection_shard2_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1093919 INFO  (parallelCoreAdminExecutor-2505-thread-1-processing-n:127.0.0.1:44117_solr x:reloaded_collection_shard2_replica_n7 0742e45f-f136-4c1c-b25e-56309b8b315026831589258516231 RELOAD) [n:127.0.0.1:44117_solr    x:reloaded_collection_shard2_replica_n7] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1093955 INFO  (qtp1381211665-8448) [n:127.0.0.

[...truncated too long message...]

pace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null271290127
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null271290127
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null271290127

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null1725005841
     [copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null1725005841
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null1725005841

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 42ms :: artifacts dl 2ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 156 minutes 20 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2533 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2533/

1 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.SearchRateTriggerTest.testTrigger

Error Message:
expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
        at __randomizedtesting.SeedInfo.seed([530F4649DF8F02FC:30C470CB464071D1]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.cloud.autoscaling.SearchRateTriggerTest.testTrigger(SearchRateTriggerTest.java:133)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13622 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.SearchRateTriggerTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.SearchRateTriggerTest_530F4649DF8F02FC-001/init-core-data-001
   [junit4]   2> 1738614 WARN  (SUITE-SearchRateTriggerTest-seed#[530F4649DF8F02FC]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 1738614 INFO  (SUITE-SearchRateTriggerTest-seed#[530F4649DF8F02FC]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1738616 INFO  (SUITE-SearchRateTriggerTest-seed#[530F4649DF8F02FC]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1738616 INFO  (SUITE-SearchRateTriggerTest-seed#[530F4649DF8F02FC]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1738617 INFO  (SUITE-SearchRateTriggerTest-seed#[530F4649DF8F02FC]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.SearchRateTriggerTest_530F4649DF8F02FC-001/tempDir-001
   [junit4]   2> 1738617 INFO  (SUITE-SearchRateTriggerTest-seed#[530F4649DF8F02FC]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1738643 INFO  (Thread-2324) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1738643 INFO  (Thread-2324) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1738663 ERROR (Thread-2324) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1738743 INFO  (SUITE-SearchRateTriggerTest-seed#[530F4649DF8F02FC]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:50797
   [junit4]   2> 1738779 INFO  (zkConnectionManagerCallback-2680-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1738789 INFO  (jetty-launcher-2677-thread-1) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 1738805 INFO  (jetty-launcher-2677-thread-3) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 1738835 INFO  (jetty-launcher-2677-thread-2) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 1738942 INFO  (jetty-launcher-2677-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1738942 INFO  (jetty-launcher-2677-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1738942 INFO  (jetty-launcher-2677-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1738955 INFO  (jetty-launcher-2677-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5f52e60f{/solr,null,AVAILABLE}
   [junit4]   2> 1738965 INFO  (jetty-launcher-2677-thread-4) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 1738999 INFO  (jetty-launcher-2677-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1738999 INFO  (jetty-launcher-2677-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1738999 INFO  (jetty-launcher-2677-thread-3) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1739000 INFO  (jetty-launcher-2677-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1c1149d1{/solr,null,AVAILABLE}
   [junit4]   2> 1739001 INFO  (jetty-launcher-2677-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5af39947{SSL,[ssl, http/1.1]}{127.0.0.1:45659}
   [junit4]   2> 1739001 INFO  (jetty-launcher-2677-thread-3) [    ] o.e.j.s.Server Started @1739122ms
   [junit4]   2> 1739001 INFO  (jetty-launcher-2677-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45659}
   [junit4]   2> 1739002 ERROR (jetty-launcher-2677-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1739002 INFO  (jetty-launcher-2677-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1739002 INFO  (jetty-launcher-2677-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1739002 INFO  (jetty-launcher-2677-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1739002 INFO  (jetty-launcher-2677-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1739002 INFO  (jetty-launcher-2677-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-18T05:37:06.734Z
   [junit4]   2> 1739165 INFO  (jetty-launcher-2677-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@398314e6{SSL,[ssl, http/1.1]}{127.0.0.1:46437}
   [junit4]   2> 1739165 INFO  (jetty-launcher-2677-thread-2) [    ] o.e.j.s.Server Started @1739285ms
   [junit4]   2> 1739165 INFO  (jetty-launcher-2677-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46437}
   [junit4]   2> 1739165 ERROR (jetty-launcher-2677-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1739165 INFO  (jetty-launcher-2677-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1739165 INFO  (jetty-launcher-2677-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1739165 INFO  (jetty-launcher-2677-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1739165 INFO  (jetty-launcher-2677-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1739165 INFO  (jetty-launcher-2677-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-18T05:37:06.897Z
   [junit4]   2> 1739233 INFO  (jetty-launcher-2677-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1739233 INFO  (jetty-launcher-2677-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1739233 INFO  (jetty-launcher-2677-thread-4) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1739233 INFO  (jetty-launcher-2677-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1739233 INFO  (jetty-launcher-2677-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1739233 INFO  (jetty-launcher-2677-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1739239 INFO  (zkConnectionManagerCallback-2682-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1739240 INFO  (jetty-launcher-2677-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1739259 INFO  (jetty-launcher-2677-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@abc9395{/solr,null,AVAILABLE}
   [junit4]   2> 1739260 INFO  (jetty-launcher-2677-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@48e60cdd{SSL,[ssl, http/1.1]}{127.0.0.1:36550}
   [junit4]   2> 1739285 INFO  (jetty-launcher-2677-thread-1) [    ] o.e.j.s.Server Started @1739406ms
   [junit4]   2> 1739285 INFO  (jetty-launcher-2677-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36550}
   [junit4]   2> 1739285 ERROR (jetty-launcher-2677-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1739285 INFO  (jetty-launcher-2677-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1739285 INFO  (jetty-launcher-2677-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1739285 INFO  (jetty-launcher-2677-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1739285 INFO  (jetty-launcher-2677-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1739285 INFO  (jetty-launcher-2677-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-18T05:37:07.017Z
   [junit4]   2> 1739283 INFO  (jetty-launcher-2677-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3adea202{/solr,null,AVAILABLE}
   [junit4]   2> 1739300 INFO  (jetty-launcher-2677-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2d66277b{SSL,[ssl, http/1.1]}{127.0.0.1:51759}
   [junit4]   2> 1739300 INFO  (jetty-launcher-2677-thread-4) [    ] o.e.j.s.Server Started @1739421ms
   [junit4]   2> 1739300 INFO  (jetty-launcher-2677-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51759}
   [junit4]   2> 1739301 ERROR (jetty-launcher-2677-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1739301 INFO  (jetty-launcher-2677-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1739301 INFO  (jetty-launcher-2677-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1739301 INFO  (jetty-launcher-2677-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1739301 INFO  (jetty-launcher-2677-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1739301 INFO  (jetty-launcher-2677-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-18T05:37:07.033Z
   [junit4]   2> 1739345 INFO  (zkConnectionManagerCallback-2684-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1739346 INFO  (jetty-launcher-2677-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1739536 INFO  (zkConnectionManagerCallback-2686-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1739537 INFO  (jetty-launcher-2677-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1739545 INFO  (jetty-launcher-2677-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50797/solr
   [junit4]   2> 1739562 INFO  (zkConnectionManagerCallback-2688-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1739563 INFO  (zkConnectionManagerCallback-2692-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1739565 INFO  (jetty-launcher-2677-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1739575 INFO  (jetty-launcher-2677-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50797/solr
   [junit4]   2> 1739631 INFO  (zkConnectionManagerCallback-2698-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1739655 INFO  (zkConnectionManagerCallback-2696-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1739656 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10641cc7a210006, likely client has closed socket
   [junit4]   2> 1739704 INFO  (jetty-launcher-2677-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50797/solr
   [junit4]   2> 1739738 INFO  (jetty-launcher-2677-thread-3) [n:127.0.0.1:45659_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1739739 INFO  (jetty-launcher-2677-thread-3) [n:127.0.0.1:45659_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45659_solr
   [junit4]   2> 1739740 INFO  (jetty-launcher-2677-thread-3) [n:127.0.0.1:45659_solr    ] o.a.s.c.Overseer Overseer (id=73818790376308743-127.0.0.1:45659_solr-n_0000000000) starting
   [junit4]   2> 1739800 INFO  (jetty-launcher-2677-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50797/solr
   [junit4]   2> 1739835 INFO  (zkConnectionManagerCallback-2704-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1739854 INFO  (zkConnectionManagerCallback-2715-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1739857 INFO  (jetty-launcher-2677-thread-3) [n:127.0.0.1:45659_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:50797/solr ready
   [junit4]   2> 1739937 INFO  (jetty-launcher-2677-thread-3) [n:127.0.0.1:45659_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45659_solr
   [junit4]   2> 1739977 INFO  (zkConnectionManagerCallback-2707-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1739993 INFO  (OverseerStateUpdate-73818790376308743-127.0.0.1:45659_solr-n_0000000000) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1740009 INFO  (jetty-launcher-2677-thread-2) [n:127.0.0.1:46437_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1740012 INFO  (jetty-launcher-2677-thread-2) [n:127.0.0.1:46437_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1740014 INFO  (jetty-launcher-2677-thread-2) [n:127.0.0.1:46437_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1740014 INFO  (jetty-launcher-2677-thread-2) [n:127.0.0.1:46437_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46437_solr
   [junit4]   2> 1740023 INFO  (zkConnectionManagerCallback-2711-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1740023 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10641cc7a21000b, likely client has closed socket
   [junit4]   2> 1740054 INFO  (zkConnectionManagerCallback-2717-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1740061 INFO  (jetty-launcher-2677-thread-1) [n:127.0.0.1:36550_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1740063 INFO  (jetty-launcher-2677-thread-1) [n:127.0.0.1:36550_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1740065 INFO  (jetty-launcher-2677-thread-1) [n:127.0.0.1:36550_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1740065 INFO  (jetty-launcher-2677-thread-1) [n:127.0.0.1:36550_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36550_solr
   [junit4]   2> 1740070 INFO  (zkCallback-2714-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1740231 INFO  (zkCallback-2716-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1740231 INFO  (zkCallback-2697-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
   [junit4]   2> 1740231 INFO  (zkCallback-2706-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
   [junit4]   2> 1740256 INFO  (zkConnectionManagerCallback-2723-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1740264 INFO  (jetty-launcher-2677-thread-4) [n:127.0.0.1:51759_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1740266 INFO  (jetty-launcher-2677-thread-4) [n:127.0.0.1:51759_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1740268 INFO  (jetty-launcher-2677-thread-4) [n:127.0.0.1:51759_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1740269 INFO  (jetty-launcher-2677-thread-4) [n:127.0.0.1:51759_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51759_solr
   [junit4]   2> 1740270 INFO  (zkCallback-2714-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1740270 INFO  (zkCallback-2706-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1740271 INFO  (zkCallback-2716-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1740271 INFO  (zkCallback-2697-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1740347 INFO  (zkCallback-2722-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1741555 INFO  (jetty-launcher-2677-thread-4) [n:127.0.0.1:51759_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51759.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6839015f
   [junit4]   2> 1741627 INFO  (jetty-launcher-2677-thread-2) [n:127.0.0.1:46437_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46437.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6839015f
   [junit4]   2> 1741669 INFO  (jetty-launcher-2677-thread-2) [n:127.0.0.1:46437_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46437.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6839015f
   [junit4]   2> 1741669 INFO  (jetty-launcher-2677-thread-2) [n:127.0.0.1:46437_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46437.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6839015f
   [junit4]   2> 1741673 INFO  (jetty-launcher-2677-thread-4) [n:127.0.0.1:51759_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51759.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6839015f
   [junit4]   2> 1741673 INFO  (jetty-launcher-2677-thread-4) [n:127.0.0.1:51759_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51759.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6839015f
   [junit4]   2> 1741691 INFO  (jetty-launcher-2677-thread-2) [n:127.0.0.1:46437_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.SearchRateTriggerTest_530F4649DF8F02FC-001/tempDir-001/node2/.
   [junit4]   2> 1741691 INFO  (jetty-launcher-2677-thread-4) [n:127.0.0.1:51759_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.SearchRateTriggerTest_530F4649DF8F02FC-001/tempDir-001/node4/.
   [junit4]   2> 1741813 INFO  (zkConnectionManagerCallback-2731-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1741817 INFO  (jetty-launcher-2677-thread-2) [n:127.0.0.1:46437_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1741818 INFO  (zkConnectionManagerCallback-2735-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1741819 INFO  (jetty-launcher-2677-thread-2) [n:127.0.0.1:46437_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:50797/solr ready
   [junit4]   2> 1741828 INFO  (jetty-launcher-2677-thread-4) [n:127.0.0.1:51759_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1741829 INFO  (jetty-launcher-2677-thread-4) [n:127.0.0.1:51759_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:50797/solr ready
   [junit4]   2> 1741843 INFO  (jetty-launcher-2677-thread-1) [n:127.0.0.1:36550_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36550.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6839015f
   [junit4]   2> 1741858 INFO  (jetty-launcher-2677-thread-3) [n:127.0.0.1:45659_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45659.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6839015f
   [junit4]   2> 1741938 INFO  (jetty-launcher-2677-thread-1) [n:127.0.0.1:36550_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36550.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6839015f
   [junit4]   2> 1741938 INFO  (jetty-launcher-2677-thread-1) [n:127.0.0.1:36550_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36550.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6839015f
   [junit4]   2> 1741976 INFO  (jetty-launcher-2677-thread-3) [n:127.0.0.1:45659_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45659.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6839015f
   [junit4]   2> 1741976 INFO  (jetty-launcher-2677-thread-3) [n:127.0.0.1:45659_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45659.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6839015f
   [junit4]   2> 1741978 INFO  (jetty-launcher-2677-thread-3) [n:127.0.0.1:45659_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.SearchRateTriggerTest_530F4649DF8F02FC-001/tempDir-001/node3/.
   [junit4]   2> 1741984 INFO  (jetty-launcher-2677-thread-1) [n:127.0.0.1:36550_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.SearchRateTriggerTest_530F4649DF8F02FC-001/tempDir-001/node1/.
   [junit4]   2> 1742028 INFO  (zkConnectionManagerCallback-2740-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1742029 INFO  (jetty-launcher-2677-thread-1) [n:127.0.0.1:36550_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1742030 INFO  (jetty-launcher-2677-thread-1) [n:127.0.0.1:36550_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:50797/solr ready
   [junit4]   2> 1742807 INFO  (zkConnectionManagerCallback-2742-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1742827 INFO  (zkConnectionManagerCallback-2747-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1742828 INFO  (SUITE-SearchRateTriggerTest-seed#[530F4649DF8F02FC]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1742829 INFO  (SUITE-SearchRateTriggerTest-seed#[530F4649DF8F02FC]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:50797/solr ready
   [junit4]   2> 1743373 INFO  (TEST-SearchRateTriggerTest.testWaitForElapsed-seed#[530F4649DF8F02FC]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testWaitForElapsed
   [junit4]   2> 1743374 INFO  (TEST-SearchRateTriggerTest.testWaitForElapsed-seed#[530F4649DF8F02FC]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1743607 INFO  (qtp924348450-10840) [n:127.0.0.1:45659_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&maxShardsPerNode=1&name=SearchRateTriggerTest-collection1&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1743623 INFO  (OverseerThreadFactory-3544-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection SearchRateTriggerTest-collection1
   [junit4]   2> 1743782 INFO  (OverseerStateUpdate-73818790376308743-127.0.0.1:45659_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"SearchRateTriggerTest-collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"SearchRateTriggerTest-collection1_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:45659/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1743786 INFO  (OverseerStateUpdate-73818790376308743-127.0.0.1:45659_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"SearchRateTriggerTest-collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"SearchRateTriggerTest-collection1_shard1_replica_n3",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:51759/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1743789 INFO  (OverseerStateUpdate-73818790376308743-127.0.0.1:45659_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"SearchRateTriggerTest-collection1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"SearchRateTriggerTest-collection1_shard2_replica_n5",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:46437/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1743793 INFO  (OverseerStateUpdate-73818790376308743-127.0.0.1:45659_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"SearchRateTriggerTest-collection1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"SearchRateTriggerTest-collection1_shard2_replica_n7",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:36550/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1744108 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr    x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=SearchRateTriggerTest-collection1_shard2_replica_n7&action=CREATE&numShards=2&collection=SearchRateTriggerTest-collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1744128 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr    x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&newCollection=true&name=SearchRateTriggerTest-collection1_shard2_replica_n5&action=CREATE&numShards=2&collection=SearchRateTriggerTest-collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1744185 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr    x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=SearchRateTriggerTest-collection1_shard1_replica_n3&action=CREATE&numShards=2&collection=SearchRateTriggerTest-collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1744186 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr    x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=SearchRateTriggerTest-collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=SearchRateTriggerTest-collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1744186 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr    x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1745153 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1745175 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1745236 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.s.IndexSchema [SearchRateTriggerTest-collection1_shard2_replica_n7] Schema name=minimal
   [junit4]   2> 1745239 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.s.IndexSchema [SearchRateTriggerTest-collection1_shard2_replica_n5] Schema name=minimal
   [junit4]   2> 1745255 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1745256 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.c.CoreContainer Creating SolrCore 'SearchRateTriggerTest-collection1_shard2_replica_n7' using configuration from collection SearchRateTriggerTest-collection1, trusted=true
   [junit4]   2> 1745256 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36550.solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7' (registry 'solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6839015f
   [junit4]   2> 1745256 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1745257 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.c.SolrCore [[SearchRateTriggerTest-collection1_shard2_replica_n7] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.SearchRateTriggerTest_530F4649DF8F02FC-001/tempDir-001/node1/SearchRateTriggerTest-collection1_shard2_replica_n7], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.SearchRateTriggerTest_530F4649DF8F02FC-001/tempDir-001/node1/./SearchRateTriggerTest-collection1_shard2_replica_n7/data/]
   [junit4]   2> 1745255 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1745257 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'SearchRateTriggerTest-collection1_shard2_replica_n5' using configuration from collection SearchRateTriggerTest-collection1, trusted=true
   [junit4]   2> 1745258 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46437.solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5' (registry 'solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6839015f
   [junit4]   2> 1745258 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1745258 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.c.SolrCore [[SearchRateTriggerTest-collection1_shard2_replica_n5] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.SearchRateTriggerTest_530F4649DF8F02FC-001/tempDir-001/node2/SearchRateTriggerTest-collection1_shard2_replica_n5], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.SearchRateTriggerTest_530F4649DF8F02FC-001/tempDir-001/node2/./SearchRateTriggerTest-collection1_shard2_replica_n5/data/]
   [junit4]   2> 1745283 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1745285 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1745304 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.s.IndexSchema [SearchRateTriggerTest-collection1_shard1_replica_n3] Schema name=minimal
   [junit4]   2> 1745309 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.s.IndexSchema [SearchRateTriggerTest-collection1_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 1745330 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1745330 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'SearchRateTriggerTest-collection1_shard1_replica_n1' using configuration from collection SearchRateTriggerTest-collection1, trusted=true
   [junit4]   2> 1745332 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1745332 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'SearchRateTriggerTest-collection1_shard1_replica_n3' using configuration from collection SearchRateTriggerTest-collection1, trusted=true
   [junit4]   2> 1745333 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51759.solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3' (registry 'solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6839015f
   [junit4]   2> 1745333 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1745333 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.c.SolrCore [[SearchRateTriggerTest-collection1_shard1_replica_n3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.SearchRateTriggerTest_530F4649DF8F02FC-001/tempDir-001/node4/SearchRateTriggerTest-collection1_shard1_replica_n3], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.SearchRateTriggerTest_530F4649DF8F02FC-001/tempDir-001/node4/./SearchRateTriggerTest-collection1_shard1_replica_n3/data/]
   [junit4]   2> 1745404 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45659.solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1' (registry 'solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6839015f
   [junit4]   2> 1745405 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1745405 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.c.SolrCore [[SearchRateTriggerTest-collection1_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.SearchRateTriggerTest_530F4649DF8F02FC-001/tempDir-001/node3/SearchRateTriggerTest-collection1_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.SearchRateTriggerTest_530F4649DF8F02FC-001/tempDir-001/node3/./SearchRateTriggerTest-collection1_shard1_replica_n1/data/]
   [junit4]   2> 1745667 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1745667 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1745698 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1745698 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1745719 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.s.SolrIndexSearcher Opening [Searcher@693e7c95[SearchRateTriggerTest-collection1_shard2_replica_n7] main]
   [junit4]   2> 1745720 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1745725 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1745726 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1745747 INFO  (searcherExecutor-3553-thread-1-processing-n:127.0.0.1:36550_solr x:SearchRateTriggerTest-collection1_shard2_replica_n7 c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.c.SolrCore [SearchRateTriggerTest-collection1_shard2_replica_n7] Registered new searcher Searcher@693e7c95[SearchRateTriggerTest-collection1_shard2_replica_n7] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1745753 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600779015668367360
   [junit4]   2> 1745972 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1745972 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1745974 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1745974 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1745978 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.c.ZkShardTerms Successful update of terms at /collections/SearchRateTriggerTest-collection1/terms/shard2 to Terms{values={core_node8=0}, version=0}
   [junit4]   2> 1745985 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9998ms
   [junit4]   2> 1745985 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@3d9cbcc[SearchRateTriggerTest-collection1_shard1_replica_n3] main]
   [junit4]   2> 1745987 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1745987 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1745988 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1745992 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600779015918977024
   [junit4]   2> 1746004 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1746004 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1746006 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1746006 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1746012 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/SearchRateTriggerTest-collection1/terms/shard1 to Terms{values={core_node4=0}, version=0}
   [junit4]   2> 1746021 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 1746035 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5db6da9a[SearchRateTriggerTest-collection1_shard1_replica_n1] main]
   [junit4]   2> 1746037 INFO  (searcherExecutor-3556-thread-1-processing-n:127.0.0.1:51759_solr x:SearchRateTriggerTest-collection1_shard1_replica_n3 c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.c.SolrCore [SearchRateTriggerTest-collection1_shard1_replica_n3] Registered new searcher Searcher@3d9cbcc[SearchRateTriggerTest-collection1_shard1_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1746038 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1746038 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1746039 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1746039 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600779015968260096
   [junit4]   2> 1746054 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/SearchRateTriggerTest-collection1/terms/shard1 to Terms{values={core_node2=0, core_node4=0}, version=1}
   [junit4]   2> 1746060 INFO  (searcherExecutor-3555-thread-1-processing-n:127.0.0.1:45659_solr x:SearchRateTriggerTest-collection1_shard1_replica_n1 c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.c.SolrCore [SearchRateTriggerTest-collection1_shard1_replica_n1] Registered new searcher Searcher@5db6da9a[SearchRateTriggerTest-collection1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1746064 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1746064 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1746065 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1746065 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1746068 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.s.SolrIndexSearcher Opening [Searcher@620a5f6[SearchRateTriggerTest-collection1_shard2_replica_n5] main]
   [junit4]   2> 1746070 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1746070 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1746071 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1746071 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600779016001814528
   [junit4]   2> 1746079 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.c.ZkShardTerms Successful update of terms at /collections/SearchRateTriggerTest-collection1/terms/shard2 to Terms{values={core_node6=0, core_node8=0}, version=1}
   [junit4]   2> 1746082 INFO  (searcherExecutor-3554-thread-1-processing-n:127.0.0.1:46437_solr x:SearchRateTriggerTest-collection1_shard2_replica_n5 c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.c.SolrCore [SearchRateTriggerTest-collection1_shard2_replica_n5] Registered new searcher Searcher@620a5f6[SearchRateTriggerTest-collection1_shard2_replica_n5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1746486 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1746486 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1746486 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:36550/solr/SearchRateTriggerTest-collection1_shard2_replica_n7/
   [junit4]   2> 1746487 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.u.PeerSync PeerSync: core=SearchRateTriggerTest-collection1_shard2_replica_n7 url=https://127.0.0.1:36550/solr START replicas=[https://127.0.0.1:46437/solr/SearchRateTriggerTest-collection1_shard2_replica_n5/] nUpdates=100
   [junit4]   2> 1746500 INFO  (qtp1091253632-10826) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.c.S.Request [SearchRateTriggerTest-collection1_shard2_replica_n5]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 1746522 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1746522 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1746522 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:51759/solr/SearchRateTriggerTest-collection1_shard1_replica_n3/
   [junit4]   2> 1746523 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=SearchRateTriggerTest-collection1_shard1_replica_n3 url=https://127.0.0.1:51759/solr START replicas=[https://127.0.0.1:45659/solr/SearchRateTriggerTest-collection1_shard1_replica_n1/] nUpdates=100
   [junit4]   2> 1746559 INFO  (qtp924348450-10836) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.c.S.Request [SearchRateTriggerTest-collection1_shard1_replica_n1]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=26
   [junit4]   2> 1746787 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.u.PeerSync PeerSync: core=SearchRateTriggerTest-collection1_shard2_replica_n7 url=https://127.0.0.1:36550/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 1746788 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 1746788 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 1746788 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1746792 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:36550/solr/SearchRateTriggerTest-collection1_shard2_replica_n7/ shard2
   [junit4]   2> 1746823 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=SearchRateTriggerTest-collection1_shard1_replica_n3 url=https://127.0.0.1:51759/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 1746823 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 1746823 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 1746823 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1746829 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:51759/solr/SearchRateTriggerTest-collection1_shard1_replica_n3/ shard1
   [junit4]   2> 1747038 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1747041 INFO  (qtp1077484287-10850) [n:127.0.0.1:36550_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node8 x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=SearchRateTriggerTest-collection1_shard2_replica_n7&action=CREATE&numShards=2&collection=SearchRateTriggerTest-collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2933
   [junit4]   2> 1747048 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1747050 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node4 x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=SearchRateTriggerTest-collection1_shard1_replica_n3&action=CREATE&numShards=2&collection=SearchRateTriggerTest-collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2865
   [junit4]   2> 1747059 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr c:SearchRateTriggerTest-collection1 s:shard1 r:core_node2 x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=SearchRateTriggerTest-collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=SearchRateTriggerTest-collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2873
   [junit4]   2> 1747088 INFO  (qtp1091253632-10827) [n:127.0.0.1:46437_solr c:SearchRateTriggerTest-collection1 s:shard2 r:core_node6 x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&newCollection=true&name=SearchRateTriggerTest-collection1_shard2_replica_n5&action=CREATE&numShards=2&collection=SearchRateTriggerTest-collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2960
   [junit4]   2> 1747093 INFO  (qtp924348450-10840) [n:127.0.0.1:45659_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1747188 INFO  (zkCallback-2706-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/SearchRateTriggerTest-collection1/state.json] for collection [SearchRateTriggerTest-collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1747188 INFO  (zkCallback-2716-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/SearchRateTriggerTest-collection1/state.json] for collection [SearchRateTriggerTest-collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1747188 INFO  (zkCallback-2722-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/SearchRateTriggerTest-collection1/state.json] for collection [SearchRateTriggerTest-collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1747189 INFO  (zkCallback-2697-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/SearchRateTriggerTest-collection1/state.json] for collection [SearchRateTriggerTest-collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1747631 INFO  (OverseerCollectionConfigSetProcessor-73818790376308743-127.0.0.1:45659_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1748095 INFO  (qtp924348450-10840) [n:127.0.0.1:45659_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&maxShardsPerNode=1&name=SearchRateTriggerTest-collection1&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=4487
   [junit4]   2> 1748113 INFO  (qtp924348450-10838) [n:127.0.0.1:45659_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1:QUERY./select.requestTimes:1minRate} status=0 QTime=7
   [junit4]   2> 1748132 INFO  (qtp1077484287-10851) [n:127.0.0.1:36550_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1748171 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3:QUERY./select.requestTimes:1minRate} status=0 QTime=6
   [junit4]   2> 1748178 INFO  (qtp1091253632-10829) [n:127.0.0.1:46437_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1748287 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1748289 INFO  (qtp1077484287-10848) [n:127.0.0.1:36550_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1748296 INFO  (qtp1281400165-10855) [n:127.0.0.1:51759_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1748304 INFO  (qtp1091253632-10830) [n:127.0.0.1:46437_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1749307 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1749308 INFO  (qtp1077484287-10848) [n:127.0.0.1:36550_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1749310 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1749311 INFO  (qtp1091253632-10829) [n:127.0.0.1:46437_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1750313 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1750315 INFO  (qtp1077484287-10848) [n:127.0.0.1:36550_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1750316 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1750317 INFO  (qtp1091253632-10829) [n:127.0.0.1:46437_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1751320 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1751321 INFO  (qtp1077484287-10848) [n:127.0.0.1:36550_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1751322 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1751325 INFO  (qtp1091253632-10829) [n:127.0.0.1:46437_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1752339 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1752424 INFO  (qtp1077484287-10848) [n:127.0.0.1:36550_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1752425 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1752427 INFO  (qtp1091253632-10829) [n:127.0.0.1:46437_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1753490 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1753492 INFO  (qtp1077484287-10848) [n:127.0.0.1:36550_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1753494 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1753495 INFO  (qtp1091253632-10829) [n:127.0.0.1:46437_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1754557 INFO  (qtp924348450-10838) [n:127.0.0.1:45659_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1754560 INFO  (qtp1077484287-10848) [n:127.0.0.1:36550_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1754568 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1754571 INFO  (qtp1091253632-10829) [n:127.0.0.1:46437_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1755581 INFO  (qtp924348450-10838) [n:127.0.0.1:45659_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1755583 INFO  (qtp1077484287-10848) [n:127.0.0.1:36550_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1755584 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1755586 INFO  (qtp1091253632-10829) [n:127.0.0.1:46437_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1755588 INFO  (qtp924348450-10838) [n:127.0.0.1:45659_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1755590 INFO  (qtp1077484287-10848) [n:127.0.0.1:36550_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1755591 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1755593 INFO  (qtp1091253632-10829) [n:127.0.0.1:46437_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1756595 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1756597 INFO  (qtp1077484287-10848) [n:127.0.0.1:36550_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1756598 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1756599 INFO  (qtp1091253632-10829) [n:127.0.0.1:46437_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1757694 INFO  (qtp924348450-10838) [n:127.0.0.1:45659_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1757697 INFO  (qtp1077484287-10848) [n:127.0.0.1:36550_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1757698 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1757704 INFO  (qtp1091253632-10829) [n:127.0.0.1:46437_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1760715 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1760721 INFO  (qtp1077484287-10848) [n:127.0.0.1:36550_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1760724 INFO  (qtp1281400165-10858) [n:127.0.0.1:51759_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1760726 INFO  (qtp1091253632-10829) [n:127.0.0.1:46437_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5:QUERY./select.requestTimes:1minRate} status=0 QTime=0
   [junit4]   2> 1760735 INFO  (TEST-SearchRateTriggerTest.testWaitForElapsed-seed#[530F4649DF8F02FC]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testWaitForElapsed
   [junit4]   2> 1761593 INFO  (TEST-SearchRateTriggerTest.testDefaultsAndBackcompat-seed#[530F4649DF8F02FC]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDefaultsAndBackcompat
   [junit4]   2> 1761594 INFO  (TEST-SearchRateTriggerTest.testDefaultsAndBackcompat-seed#[530F4649DF8F02FC]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1761601 INFO  (qtp924348450-10839) [n:127.0.0.1:45659_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=SearchRateTriggerTest-collection1&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1761640 INFO  (OverseerThreadFactory-3544-thread-2) [    ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd=action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true, asyncId=null
   [junit4]   2> 1761791 INFO  (qtp1281400165-10856) [n:127.0.0.1:51759_solr    x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3, tag=null
   [junit4]   2> 1761791 INFO  (qtp1281400165-10856) [n:127.0.0.1:51759_solr    x:SearchRateTriggerTest-collection1_shard1_replica_n3] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@70e99243: rootName = solr_51759, domain = solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3, service url = null, agent id = null] for registry solr.core.SearchRateTriggerTest-collection1.shard1.replica_n3 / com.codahale.metrics.MetricRegistry@3d6b0101
   [junit4]   2> 1761791 INFO  (qtp1091253632-10829) [n:127.0.0.1:46437_solr    x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5, tag=null
   [junit4]   2> 1761791 INFO  (qtp1091253632-10829) [n:127.0.0.1:46437_solr    x:SearchRateTriggerTest-collection1_shard2_replica_n5] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4b521c71: rootName = solr_46437, domain = solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5, service url = null, agent id = null] for registry solr.core.SearchRateTriggerTest-collection1.shard2.replica_n5 / com.codahale.metrics.MetricRegistry@7b68a7ef
   [junit4]   2> 1761792 INFO  (qtp924348450-10842) [n:127.0.0.1:45659_solr    x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1, tag=null
   [junit4]   2> 1761792 INFO  (qtp924348450-10842) [n:127.0.0.1:45659_solr    x:SearchRateTriggerTest-collection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@9b5dad4: rootName = solr_45659, domain = solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.SearchRateTriggerTest-collection1.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@4003f30
   [junit4]   2> 1761799 INFO  (qtp1077484287-10847) [n:127.0.0.1:36550_solr    x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7, tag=null
   [junit4]   2> 1761799 INFO  (qtp1077484287-10847) [n:127.0.0.1:36550_solr    x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2cb52438: rootName = solr_36550, domain = solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7, service url = null, agent id = null] for registry solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7 / com.codahale.metrics.MetricRegistry@6d5e003d
   [junit4]   2> 1762075 INFO  (qtp1077484287-10847) [n:127.0.0.1:36550_solr    x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.c.SolrCore [SearchRateTriggerTest-collection1_shard2_replica_n7]  CLOSING SolrCore org.apache.solr.core.SolrCore@19e989ed
   [junit4]   2> 1762075 INFO  (qtp1077484287-10847) [n:127.0.0.1:36550_solr    x:SearchRateTriggerTest-collection1_shard2_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.SearchRateTriggerTest-collection1.shard2.replica_n7, tag=19e989ed
   [junit4]   2> 1762075 INFO  (qtp1077484287-10847) [n:127.0.0.1:36550_solr    x:SearchRateTri

[...truncated too long message...]

pace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null1582835298
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null1582835298
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null1582835298

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null897355973
     [copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null897355973
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null897355973

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 58ms :: artifacts dl 9ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 159 minutes 22 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2534 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2534/

2 tests failed.
FAILED:  org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica

Error Message:
Expected 1x2 collections null Live Nodes: [127.0.0.1:35004_solr, 127.0.0.1:36484_solr, 127.0.0.1:42582_solr, 127.0.0.1:45646_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Expected 1x2 collections
null
Live Nodes: [127.0.0.1:35004_solr, 127.0.0.1:36484_solr, 127.0.0.1:42582_solr, 127.0.0.1:45646_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([5C9A2B9FC62A5C92:368C4A4FAED81658]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:278)
        at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:235)
        at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:226)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure

Error Message:
Did not see a fully active cluster after 30 seconds

Stack Trace:
java.lang.AssertionError: Did not see a fully active cluster after 30 seconds
        at __randomizedtesting.SeedInfo.seed([E43EDFE115CD53C4:6C087DB2CD62BBD6]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure(TestCollectionStateWatchers.java:250)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12626 lines...]
   [junit4] Suite: org.apache.solr.cloud.DeleteReplicaTest
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_5C9A2B9FC62A5C92-001/init-core-data-001
   [junit4]   2> 438940 WARN  (SUITE-DeleteReplicaTest-seed#[5C9A2B9FC62A5C92]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 438942 INFO  (SUITE-DeleteReplicaTest-seed#[5C9A2B9FC62A5C92]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 438944 INFO  (SUITE-DeleteReplicaTest-seed#[5C9A2B9FC62A5C92]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 438944 INFO  (SUITE-DeleteReplicaTest-seed#[5C9A2B9FC62A5C92]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 438947 INFO  (SUITE-DeleteReplicaTest-seed#[5C9A2B9FC62A5C92]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_5C9A2B9FC62A5C92-001/tempDir-001
   [junit4]   2> 438947 INFO  (SUITE-DeleteReplicaTest-seed#[5C9A2B9FC62A5C92]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 438970 INFO  (Thread-115) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 438970 INFO  (Thread-115) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 439115 INFO  (SUITE-DeleteReplicaTest-seed#[5C9A2B9FC62A5C92]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:41408
   [junit4]   2> 439116 ERROR (Thread-115) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 439429 INFO  (zkConnectionManagerCallback-268-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 439446 INFO  (jetty-launcher-265-thread-1) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 439462 INFO  (jetty-launcher-265-thread-2) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 439483 INFO  (jetty-launcher-265-thread-3) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 439519 INFO  (jetty-launcher-265-thread-4) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 439628 INFO  (jetty-launcher-265-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 439628 INFO  (jetty-launcher-265-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 439628 INFO  (jetty-launcher-265-thread-3) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 439629 INFO  (jetty-launcher-265-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5a7ec169{/solr,null,AVAILABLE}
   [junit4]   2> 439720 INFO  (jetty-launcher-265-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 439720 INFO  (jetty-launcher-265-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 439721 INFO  (jetty-launcher-265-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 439754 INFO  (jetty-launcher-265-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5f9f7ef7{/solr,null,AVAILABLE}
   [junit4]   2> 439900 INFO  (jetty-launcher-265-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 439900 INFO  (jetty-launcher-265-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 439901 INFO  (jetty-launcher-265-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 439924 INFO  (jetty-launcher-265-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@356834e1{HTTP/1.1,[http/1.1]}{127.0.0.1:35004}
   [junit4]   2> 439924 INFO  (jetty-launcher-265-thread-2) [    ] o.e.j.s.Server Started @440217ms
   [junit4]   2> 439924 INFO  (jetty-launcher-265-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35004}
   [junit4]   2> 439924 ERROR (jetty-launcher-265-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 439924 INFO  (jetty-launcher-265-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 439959 INFO  (jetty-launcher-265-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5a83a301{HTTP/1.1,[http/1.1]}{127.0.0.1:36484}
   [junit4]   2> 439959 INFO  (jetty-launcher-265-thread-3) [    ] o.e.j.s.Server Started @440252ms
   [junit4]   2> 439959 INFO  (jetty-launcher-265-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36484}
   [junit4]   2> 439959 ERROR (jetty-launcher-265-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 439959 INFO  (jetty-launcher-265-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 439959 INFO  (jetty-launcher-265-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 439973 INFO  (jetty-launcher-265-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 439973 INFO  (jetty-launcher-265-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 439974 INFO  (jetty-launcher-265-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-19T02:11:52.777Z
   [junit4]   2> 439974 INFO  (jetty-launcher-265-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@36b77a1b{/solr,null,AVAILABLE}
   [junit4]   2> 439973 INFO  (jetty-launcher-265-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 439975 INFO  (jetty-launcher-265-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 439975 INFO  (jetty-launcher-265-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 439975 INFO  (jetty-launcher-265-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-19T02:11:52.778Z
   [junit4]   2> 439994 INFO  (jetty-launcher-265-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 439994 INFO  (jetty-launcher-265-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 439994 INFO  (jetty-launcher-265-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@310e18f8{HTTP/1.1,[http/1.1]}{127.0.0.1:42582}
   [junit4]   2> 439994 INFO  (jetty-launcher-265-thread-4) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 439994 INFO  (jetty-launcher-265-thread-1) [    ] o.e.j.s.Server Started @440287ms
   [junit4]   2> 439994 INFO  (jetty-launcher-265-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42582}
   [junit4]   2> 439995 ERROR (jetty-launcher-265-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 439995 INFO  (jetty-launcher-265-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@55e5d691{/solr,null,AVAILABLE}
   [junit4]   2> 439995 INFO  (jetty-launcher-265-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 439995 INFO  (jetty-launcher-265-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 439995 INFO  (jetty-launcher-265-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 439995 INFO  (jetty-launcher-265-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 439995 INFO  (jetty-launcher-265-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-19T02:11:52.798Z
   [junit4]   2> 439995 INFO  (jetty-launcher-265-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@139aa117{HTTP/1.1,[http/1.1]}{127.0.0.1:45646}
   [junit4]   2> 439995 INFO  (jetty-launcher-265-thread-4) [    ] o.e.j.s.Server Started @440288ms
   [junit4]   2> 439995 INFO  (jetty-launcher-265-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45646}
   [junit4]   2> 439996 ERROR (jetty-launcher-265-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 439996 INFO  (jetty-launcher-265-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 439996 INFO  (jetty-launcher-265-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 439996 INFO  (jetty-launcher-265-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 439996 INFO  (jetty-launcher-265-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 439996 INFO  (jetty-launcher-265-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-19T02:11:52.799Z
   [junit4]   2> 440124 INFO  (zkConnectionManagerCallback-272-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 440165 INFO  (zkConnectionManagerCallback-276-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 440165 INFO  (zkConnectionManagerCallback-274-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 440201 INFO  (zkConnectionManagerCallback-270-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 440215 INFO  (jetty-launcher-265-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 440215 INFO  (jetty-launcher-265-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 440215 INFO  (jetty-launcher-265-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 440559 INFO  (jetty-launcher-265-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 440654 INFO  (jetty-launcher-265-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41408/solr
   [junit4]   2> 440675 INFO  (jetty-launcher-265-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41408/solr
   [junit4]   2> 440726 INFO  (zkConnectionManagerCallback-280-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 440765 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1021a6d69e90005, likely client has closed socket
   [junit4]   2> 440785 INFO  (jetty-launcher-265-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41408/solr
   [junit4]   2> 440856 INFO  (zkConnectionManagerCallback-284-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 440874 INFO  (jetty-launcher-265-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41408/solr
   [junit4]   2> 440911 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1021a6d69e90006, likely client has closed socket
   [junit4]   2> 441002 INFO  (zkConnectionManagerCallback-292-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 441019 INFO  (zkConnectionManagerCallback-288-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 441035 INFO  (zkConnectionManagerCallback-294-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 441055 INFO  (zkConnectionManagerCallback-296-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 441181 INFO  (zkConnectionManagerCallback-298-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 441241 INFO  (zkConnectionManagerCallback-300-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 441824 INFO  (jetty-launcher-265-thread-1) [n:127.0.0.1:42582_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 441826 INFO  (jetty-launcher-265-thread-1) [n:127.0.0.1:42582_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42582_solr
   [junit4]   2> 441845 INFO  (jetty-launcher-265-thread-1) [n:127.0.0.1:42582_solr    ] o.a.s.c.Overseer Overseer (id=72649601221984265-127.0.0.1:42582_solr-n_0000000000) starting
   [junit4]   2> 442336 INFO  (zkConnectionManagerCallback-313-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 442581 INFO  (jetty-launcher-265-thread-1) [n:127.0.0.1:42582_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41408/solr ready
   [junit4]   2> 442738 INFO  (jetty-launcher-265-thread-1) [n:127.0.0.1:42582_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42582_solr
   [junit4]   2> 442842 INFO  (zkCallback-293-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 442974 INFO  (zkCallback-312-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 444510 INFO  (jetty-launcher-265-thread-1) [n:127.0.0.1:42582_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42582.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 444603 INFO  (jetty-launcher-265-thread-1) [n:127.0.0.1:42582_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42582.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 444604 INFO  (jetty-launcher-265-thread-1) [n:127.0.0.1:42582_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42582.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 444605 INFO  (jetty-launcher-265-thread-1) [n:127.0.0.1:42582_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_5C9A2B9FC62A5C92-001/tempDir-001/node1/.
   [junit4]   2> 445352 INFO  (jetty-launcher-265-thread-4) [n:127.0.0.1:45646_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 445355 INFO  (jetty-launcher-265-thread-4) [n:127.0.0.1:45646_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 445374 INFO  (jetty-launcher-265-thread-4) [n:127.0.0.1:45646_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 445374 INFO  (jetty-launcher-265-thread-4) [n:127.0.0.1:45646_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45646_solr
   [junit4]   2> 445388 INFO  (zkCallback-312-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 445485 INFO  (zkCallback-293-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 445501 INFO  (zkCallback-297-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 449227 INFO  (jetty-launcher-265-thread-3) [n:127.0.0.1:36484_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 449262 INFO  (jetty-launcher-265-thread-2) [n:127.0.0.1:35004_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 449264 INFO  (jetty-launcher-265-thread-2) [n:127.0.0.1:35004_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 449298 INFO  (jetty-launcher-265-thread-2) [n:127.0.0.1:35004_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 449298 INFO  (jetty-launcher-265-thread-2) [n:127.0.0.1:35004_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35004_solr
   [junit4]   2> 449299 INFO  (jetty-launcher-265-thread-3) [n:127.0.0.1:36484_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 449319 INFO  (zkCallback-312-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 449343 INFO  (zkCallback-293-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 449380 INFO  (zkCallback-297-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 449592 INFO  (jetty-launcher-265-thread-3) [n:127.0.0.1:36484_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 449592 INFO  (jetty-launcher-265-thread-3) [n:127.0.0.1:36484_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36484_solr
   [junit4]   2> 449592 INFO  (zkCallback-295-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 449592 INFO  (zkCallback-299-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 450200 INFO  (zkCallback-295-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 450200 INFO  (zkCallback-297-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 450200 INFO  (zkCallback-312-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 450200 INFO  (zkCallback-299-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 450222 INFO  (zkCallback-293-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 450235 INFO  (jetty-launcher-265-thread-4) [n:127.0.0.1:45646_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45646.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 450271 INFO  (jetty-launcher-265-thread-4) [n:127.0.0.1:45646_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45646.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 450271 INFO  (jetty-launcher-265-thread-4) [n:127.0.0.1:45646_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45646.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 450272 INFO  (jetty-launcher-265-thread-4) [n:127.0.0.1:45646_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_5C9A2B9FC62A5C92-001/tempDir-001/node4/.
   [junit4]   2> 450454 INFO  (zkConnectionManagerCallback-318-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 450545 INFO  (jetty-launcher-265-thread-4) [n:127.0.0.1:45646_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 450546 INFO  (jetty-launcher-265-thread-4) [n:127.0.0.1:45646_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41408/solr ready
   [junit4]   2> 451154 INFO  (jetty-launcher-265-thread-2) [n:127.0.0.1:35004_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35004.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 451191 INFO  (jetty-launcher-265-thread-2) [n:127.0.0.1:35004_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35004.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 451192 INFO  (jetty-launcher-265-thread-2) [n:127.0.0.1:35004_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35004.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 451193 INFO  (jetty-launcher-265-thread-2) [n:127.0.0.1:35004_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_5C9A2B9FC62A5C92-001/tempDir-001/node2/.
   [junit4]   2> 451333 INFO  (zkConnectionManagerCallback-323-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 451351 INFO  (jetty-launcher-265-thread-2) [n:127.0.0.1:35004_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 451370 INFO  (jetty-launcher-265-thread-2) [n:127.0.0.1:35004_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41408/solr ready
   [junit4]   2> 453421 INFO  (jetty-launcher-265-thread-3) [n:127.0.0.1:36484_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36484.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 453530 INFO  (jetty-launcher-265-thread-3) [n:127.0.0.1:36484_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36484.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 453531 INFO  (jetty-launcher-265-thread-3) [n:127.0.0.1:36484_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36484.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 453532 INFO  (jetty-launcher-265-thread-3) [n:127.0.0.1:36484_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_5C9A2B9FC62A5C92-001/tempDir-001/node3/.
   [junit4]   2> 453712 INFO  (zkConnectionManagerCallback-328-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 453818 INFO  (jetty-launcher-265-thread-3) [n:127.0.0.1:36484_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 453851 INFO  (jetty-launcher-265-thread-3) [n:127.0.0.1:36484_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41408/solr ready
   [junit4]   2> 454187 INFO  (zkConnectionManagerCallback-330-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 454341 INFO  (zkConnectionManagerCallback-335-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 454343 INFO  (SUITE-DeleteReplicaTest-seed#[5C9A2B9FC62A5C92]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 454344 INFO  (SUITE-DeleteReplicaTest-seed#[5C9A2B9FC62A5C92]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41408/solr ready
   [junit4]   2> 454585 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.SolrTestCaseJ4 ###Starting raceConditionOnDeleteAndRegisterReplica
   [junit4]   2> 454637 INFO  (qtp550020128-786) [n:127.0.0.1:35004_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params val=true&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 454694 INFO  (qtp550020128-786) [n:127.0.0.1:35004_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={val=true&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2} status=0 QTime=57
   [junit4]   2> 454802 INFO  (qtp550020128-782) [n:127.0.0.1:35004_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=raceDeleteReplica_true&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 454870 INFO  (OverseerThreadFactory-243-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection raceDeleteReplica_true
   [junit4]   2> 455309 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr    x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=raceDeleteReplica_true_shard1_replica_n1&action=CREATE&numShards=1&collection=raceDeleteReplica_true&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 455374 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr    x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=raceDeleteReplica_true_shard1_replica_n2&action=CREATE&numShards=1&collection=raceDeleteReplica_true&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 456549 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 456621 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 456660 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.s.IndexSchema [raceDeleteReplica_true_shard1_replica_n2] Schema name=minimal
   [junit4]   2> 456926 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 456926 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'raceDeleteReplica_true_shard1_replica_n2' using configuration from collection raceDeleteReplica_true, trusted=true
   [junit4]   2> 456926 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45646.solr.core.raceDeleteReplica_true.shard1.replica_n2' (registry 'solr.core.raceDeleteReplica_true.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 456927 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 456927 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.SolrCore [[raceDeleteReplica_true_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_5C9A2B9FC62A5C92-001/tempDir-001/node4/raceDeleteReplica_true_shard1_replica_n2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_5C9A2B9FC62A5C92-001/tempDir-001/node4/./raceDeleteReplica_true_shard1_replica_n2/data/]
   [junit4]   2> 457037 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.s.IndexSchema [raceDeleteReplica_true_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 457109 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 457109 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'raceDeleteReplica_true_shard1_replica_n1' using configuration from collection raceDeleteReplica_true, trusted=true
   [junit4]   2> 457109 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35004.solr.core.raceDeleteReplica_true.shard1.replica_n1' (registry 'solr.core.raceDeleteReplica_true.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 457110 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 457110 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.c.SolrCore [[raceDeleteReplica_true_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_5C9A2B9FC62A5C92-001/tempDir-001/node2/raceDeleteReplica_true_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_5C9A2B9FC62A5C92-001/tempDir-001/node2/./raceDeleteReplica_true_shard1_replica_n1/data/]
   [junit4]   2> 457710 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 457710 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 457711 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 457712 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 457730 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 457730 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 457731 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 457731 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 457747 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@1d545264[raceDeleteReplica_true_shard1_replica_n2] main]
   [junit4]   2> 457749 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@680bb437[raceDeleteReplica_true_shard1_replica_n1] main]
   [junit4]   2> 457756 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 457756 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 457757 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 457757 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600856712082882560
   [junit4]   2> 457877 INFO  (searcherExecutor-252-thread-1-processing-n:127.0.0.1:45646_solr x:raceDeleteReplica_true_shard1_replica_n2 c:raceDeleteReplica_true s:shard1) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.SolrCore [raceDeleteReplica_true_shard1_replica_n2] Registered new searcher Searcher@1d545264[raceDeleteReplica_true_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 457924 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/raceDeleteReplica_true/terms/shard1 to Terms{values={core_node4=0}, version=0}
   [junit4]   2> 457950 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 457951 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 457951 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 457951 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600856712286306304
   [junit4]   2> 458003 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 458003 INFO  (searcherExecutor-253-thread-1-processing-n:127.0.0.1:35004_solr x:raceDeleteReplica_true_shard1_replica_n1 c:raceDeleteReplica_true s:shard1) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.c.SolrCore [raceDeleteReplica_true_shard1_replica_n1] Registered new searcher Searcher@680bb437[raceDeleteReplica_true_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 458034 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/raceDeleteReplica_true/terms/shard1 to Terms{values={core_node3=0, core_node4=0}, version=1}
   [junit4]   2> 458624 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 458624 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 458624 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45646/solr/raceDeleteReplica_true_shard1_replica_n2/
   [junit4]   2> 458661 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=raceDeleteReplica_true_shard1_replica_n2 url=http://127.0.0.1:45646/solr START replicas=[http://127.0.0.1:35004/solr/raceDeleteReplica_true_shard1_replica_n1/] nUpdates=100
   [junit4]   2> 458746 INFO  (qtp550020128-785) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.c.S.Request [raceDeleteReplica_true_shard1_replica_n1]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=71
   [junit4]   2> 458978 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=raceDeleteReplica_true_shard1_replica_n2 url=http://127.0.0.1:45646/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 458979 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 458979 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 458979 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 458982 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45646/solr/raceDeleteReplica_true_shard1_replica_n2/ shard1
   [junit4]   2> 459179 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 459194 INFO  (qtp550020128-780) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=raceDeleteReplica_true_shard1_replica_n1&action=CREATE&numShards=1&collection=raceDeleteReplica_true&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3885
   [junit4]   2> 459288 INFO  (qtp1355906168-803) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=raceDeleteReplica_true_shard1_replica_n2&action=CREATE&numShards=1&collection=raceDeleteReplica_true&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3914
   [junit4]   2> 459398 INFO  (zkCallback-299-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/raceDeleteReplica_true/state.json] for collection [raceDeleteReplica_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 459406 INFO  (zkCallback-297-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/raceDeleteReplica_true/state.json] for collection [raceDeleteReplica_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 459598 INFO  (qtp550020128-782) [n:127.0.0.1:35004_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 459598 INFO  (qtp550020128-782) [n:127.0.0.1:35004_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&name=raceDeleteReplica_true&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=4795
   [junit4]   2> 459670 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@356834e1{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 459772 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=634149545
   [junit4]   2> 459772 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 459772 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3fe62d42: rootName = solr_35004, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@6f6623b9
   [junit4]   2> 459839 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 459839 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5fcbe492: rootName = solr_35004, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@714bbe6f
   [junit4]   2> 459912 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 459912 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@145be3e8: rootName = solr_35004, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1cb37409
   [junit4]   2> 459968 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:35004_solr
   [junit4]   2> 459969 INFO  (zkCallback-322-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 459969 INFO  (zkCallback-293-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 459969 INFO  (zkCallback-317-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 459970 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 459980 INFO  (zkCallback-327-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 459980 INFO  (zkCallback-334-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 459980 INFO  (zkCallback-295-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 459980 INFO  (zkCallback-312-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 459983 INFO  (zkCallback-299-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 460044 INFO  (zkCallback-297-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 460056 INFO  (coreCloseExecutor-262-thread-1) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.c.SolrCore [raceDeleteReplica_true_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@5af48842
   [junit4]   2> 460056 INFO  (coreCloseExecutor-262-thread-1) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.raceDeleteReplica_true.shard1.replica_n1, tag=5af48842
   [junit4]   2> 460056 INFO  (coreCloseExecutor-262-thread-1) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@347b87e: rootName = solr_35004, domain = solr.core.raceDeleteReplica_true.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.raceDeleteReplica_true.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@69107055
   [junit4]   2> 460093 INFO  (coreCloseExecutor-262-thread-1) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.raceDeleteReplica_true.shard1.leader, tag=5af48842
   [junit4]   2> 460113 INFO  (zkCallback-299-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/raceDeleteReplica_true/state.json] for collection [raceDeleteReplica_true] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 460117 INFO  (zkCallback-297-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/raceDeleteReplica_true/state.json] for collection [raceDeleteReplica_true] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 460262 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5f9f7ef7{/solr,null,UNAVAILABLE}
   [junit4]   2> 460262 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 460293 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 460329 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 460329 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 460329 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 460345 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@575516d7{/solr,null,AVAILABLE}
   [junit4]   2> 460346 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@68f7f6db{HTTP/1.1,[http/1.1]}{127.0.0.1:35004}
   [junit4]   2> 460346 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.e.j.s.Server Started @460639ms
   [junit4]   2> 460346 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35004}
   [junit4]   2> 460346 ERROR (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 460361 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 460362 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 460362 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 460362 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 460362 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-19T02:12:13.165Z
   [junit4]   2> 460418 INFO  (zkConnectionManagerCallback-337-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 460496 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 460691 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41408/solr
   [junit4]   2> 460703 INFO  (zkConnectionManagerCallback-341-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 460751 INFO  (zkConnectionManagerCallback-343-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 460794 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [n:127.0.0.1:35004_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 460810 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [n:127.0.0.1:35004_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 460830 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [n:127.0.0.1:35004_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 460830 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [n:127.0.0.1:35004_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35004_solr
   [junit4]   2> 460846 INFO  (zkCallback-317-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 460846 INFO  (zkCallback-297-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 460846 INFO  (zkCallback-293-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 460866 INFO  (zkCallback-334-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 460914 INFO  (zkCallback-295-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 460914 INFO  (zkCallback-312-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 460914 INFO  (zkCallback-327-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 460918 INFO  (zkCallback-342-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 460935 INFO  (zkCallback-297-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/raceDeleteReplica_true/state.json] for collection [raceDeleteReplica_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 461065 INFO  (OverseerCollectionConfigSetProcessor-72649601221984265-127.0.0.1:42582_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 461889 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [n:127.0.0.1:35004_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35004.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 461945 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [n:127.0.0.1:35004_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35004.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 461946 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [n:127.0.0.1:35004_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35004.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 461963 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [n:127.0.0.1:35004_solr    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_5C9A2B9FC62A5C92-001/tempDir-001/node2/.
   [junit4]   2> 461963 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [n:127.0.0.1:35004_solr    ] o.a.s.c.CorePropertiesLocator Cores are: [raceDeleteReplica_true_shard1_replica_n1]
   [junit4]   2> 462104 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 462118 INFO  (zkConnectionManagerCallback-350-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 462192 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [n:127.0.0.1:35004_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 462193 INFO  (TEST-DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica-seed#[5C9A2B9FC62A5C92]) [n:127.0.0.1:35004_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41408/solr ready
   [junit4]   2> 462286 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr    ] o.a.s.s.IndexSchema [raceDeleteReplica_true_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 462300 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 462554 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 462586 INFO  (zkCallback-297-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/raceDeleteReplica_true/state.json] for collection [raceDeleteReplica_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 462602 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.s.IndexSchema [raceDeleteReplica_true_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 462604 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 462604 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'raceDeleteReplica_true_shard1_replica_n1' using configuration from collection raceDeleteReplica_true, trusted=true
   [junit4]   2> 462604 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35004.solr.core.raceDeleteReplica_true.shard1.replica_n1' (registry 'solr.core.raceDeleteReplica_true.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 462605 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 462605 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.c.SolrCore [[raceDeleteReplica_true_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_5C9A2B9FC62A5C92-001/tempDir-001/node2/raceDeleteReplica_true_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_5C9A2B9FC62A5C92-001/tempDir-001/node2/./raceDeleteReplica_true_shard1_replica_n1/data/]
   [junit4]   2> 463004 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 463004 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 463005 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 463005 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 463008 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6ffacb[raceDeleteReplica_true_shard1_replica_n1] main]
   [junit4]   2> 463025 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 463026 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 463026 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 463027 INFO  (searcherExecutor-270-thread-1-processing-n:127.0.0.1:35004_solr x:raceDeleteReplica_true_shard1_replica_n1 c:raceDeleteReplica_true s:shard1 r:core_node3) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.c.SolrCore [raceDeleteReplica_true_shard1_replica_n1] Registered new searcher Searcher@6ffacb[raceDeleteReplica_true_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 463041 INFO  (coreLoadExecutor-269-thread-1-processing-n:127.0.0.1:35004_solr) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600856717623558144
   [junit4]   2> 463061 INFO  (coreZkRegister-264-thread-1-processing-n:127.0.0.1:35004_solr x:raceDeleteReplica_true_shard1_replica_n1 c:raceDeleteReplica_true s:shard1 r:core_node3) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true s:shard1 r:core_node3 x:raceDeleteReplica_true_shard1_replica_n1] o.a.s.c.DeleteReplicaTest Running delete core CoreDescriptor[name=raceDeleteReplica_true_shard1_replica_n1;instanceDir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_5C9A2B9FC62A5C92-001/tempDir-001/node2/./raceDeleteReplica_true_shard1_replica_n1]
   [junit4]   2> 463297 INFO  (zkCallback-297-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/raceDeleteReplica_true/state.json] for collection [raceDeleteReplica_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 463773 INFO  (qtp1469723742-924) [n:127.0.0.1:35004_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params action=ADDREPLICA&collection=raceDeleteReplica_true&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 463787 INFO  (OverseerThreadFactory-243-thread-2) [ c:raceDeleteReplica_true s:shard1  ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:42582_solr for creating new replica
   [junit4]   2> 463915 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr    x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&name=raceDeleteReplica_true_shard1_replica_n5&action=CREATE&collection=raceDeleteReplica_true&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 463916 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr    x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 464041 INFO  (zkCallback-342-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/raceDeleteReplica_true/state.json] for collection [raceDeleteReplica_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 464044 INFO  (zkCallback-297-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/raceDeleteReplica_true/state.json] for collection [raceDeleteReplica_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 465154 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 465179 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.s.IndexSchema [raceDeleteReplica_true_shard1_replica_n5] Schema name=minimal
   [junit4]   2> 465215 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 465215 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'raceDeleteReplica_true_shard1_replica_n5' using configuration from collection raceDeleteReplica_true, trusted=true
   [junit4]   2> 465215 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42582.solr.core.raceDeleteReplica_true.shard1.replica_n5' (registry 'solr.core.raceDeleteReplica_true.shard1.replica_n5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67ff068b
   [junit4]   2> 465216 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 465216 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.c.SolrCore [[raceDeleteReplica_true_shard1_replica_n5] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_5C9A2B9FC62A5C92-001/tempDir-001/node1/raceDeleteReplica_true_shard1_replica_n5], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest_5C9A2B9FC62A5C92-001/tempDir-001/node1/./raceDeleteReplica_true_shard1_replica_n5/data/]
   [junit4]   2> 465743 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 465743 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 465744 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 465744 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 465746 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.s.SolrIndexSearcher Opening [Searcher@1fbebeda[raceDeleteReplica_true_shard1_replica_n5] main]
   [junit4]   2> 465760 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 465761 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 465761 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 465762 INFO  (searcherExecutor-275-thread-1-processing-n:127.0.0.1:42582_solr x:raceDeleteReplica_true_shard1_replica_n5 c:raceDeleteReplica_true s:shard1) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.c.SolrCore [raceDeleteReplica_true_shard1_replica_n5] Registered new searcher Searcher@1fbebeda[raceDeleteReplica_true_shard1_replica_n5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 465816 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600856720533356544
   [junit4]   2> 465844 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.c.ZkShardTerms Successful update of terms at /collections/raceDeleteReplica_true/terms/shard1 to Terms{values={core_node6=0, core_node3=0, core_node4=0}, version=2}
   [junit4]   2> 465854 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.c.ZkController Core needs to recover:raceDeleteReplica_true_shard1_replica_n5
   [junit4]   2> 465855 INFO  (updateExecutor-277-thread-1-processing-n:127.0.0.1:42582_solr x:raceDeleteReplica_true_shard1_replica_n5 c:raceDeleteReplica_true s:shard1) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1 r:core_node6 x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 465855 INFO  (recoveryExecutor-278-thread-1-processing-n:127.0.0.1:42582_solr x:raceDeleteReplica_true_shard1_replica_n5 c:raceDeleteReplica_true s:shard1 r:core_node6) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1 r:core_node6 x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 465855 INFO  (recoveryExecutor-278-thread-1-processing-n:127.0.0.1:42582_solr x:raceDeleteReplica_true_shard1_replica_n5 c:raceDeleteReplica_true s:shard1 r:core_node6) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1 r:core_node6 x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 465856 INFO  (qtp316791404-799) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1  x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&name=raceDeleteReplica_true_shard1_replica_n5&action=CREATE&collection=raceDeleteReplica_true&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1941
   [junit4]   2> 465882 INFO  (qtp1469723742-924) [n:127.0.0.1:35004_solr c:raceDeleteReplica_true   ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=ADDREPLICA&collection=raceDeleteReplica_true&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=2108
   [junit4]   2> 465927 INFO  (qtp1355906168-790) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1 r:core_node4 x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.S.Request [raceDeleteReplica_true_shard1_replica_n2]  webapp=/solr path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 465927 INFO  (qtp1355906168-790) [n:127.0.0.1:45646_solr c:raceDeleteReplica_true s:shard1 r:core_node4 x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.c.S.Request [raceDeleteReplica_true_shard1_replica_n2]  webapp=/solr path=/admin/ping params={wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 465931 INFO  (recoveryExecutor-278-thread-1-processing-n:127.0.0.1:42582_solr x:raceDeleteReplica_true_shard1_replica_n5 c:raceDeleteReplica_true s:shard1 r:core_node6) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1 r:core_node6 x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[raceDeleteReplica_true_shard1_replica_n5]
   [junit4]   2> 465931 INFO  (recoveryExecutor-278-thread-1-processing-n:127.0.0.1:42582_solr x:raceDeleteReplica_true_shard1_replica_n5 c:raceDeleteReplica_true s:shard1 r:core_node6) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1 r:core_node6 x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 465931 INFO  (recoveryExecutor-278-thread-1-processing-n:127.0.0.1:42582_solr x:raceDeleteReplica_true_shard1_replica_n5 c:raceDeleteReplica_true s:shard1 r:core_node6) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1 r:core_node6 x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.c.RecoveryStrategy Publishing state of core [raceDeleteReplica_true_shard1_replica_n5] as recovering, leader is [http://127.0.0.1:45646/solr/raceDeleteReplica_true_shard1_replica_n2/] and I am [http://127.0.0.1:42582/solr/raceDeleteReplica_true_shard1_replica_n5/]
   [junit4]   2> 465933 INFO  (recoveryExecutor-278-thread-1-processing-n:127.0.0.1:42582_solr x:raceDeleteReplica_true_shard1_replica_n5 c:raceDeleteReplica_true s:shard1 r:core_node6) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1 r:core_node6 x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.c.ZkShardTerms Successful update of terms at /collections/raceDeleteReplica_true/terms/shard1 to Terms{values={core_node6=0, core_node6_recovering=0, core_node3=0, core_node4=0}, version=3}
   [junit4]   2> 465972 INFO  (recoveryExecutor-278-thread-1-processing-n:127.0.0.1:42582_solr x:raceDeleteReplica_true_shard1_replica_n5 c:raceDeleteReplica_true s:shard1 r:core_node6) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1 r:core_node6 x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:45646/solr]; [WaitForState: action=PREPRECOVERY&core=raceDeleteReplica_true_shard1_replica_n2&nodeName=127.0.0.1:42582_solr&coreNodeName=core_node6&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 465981 INFO  (qtp1355906168-791) [n:127.0.0.1:45646_solr    x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node6, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
   [junit4]   2> 466000 INFO  (qtp1355906168-791) [n:127.0.0.1:45646_solr    x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=raceDeleteReplica_true, shard=shard1, thisCore=raceDeleteReplica_true_shard1_replica_n2, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:42582_solr, coreNodeName=core_node6, onlyIfActiveCheckResult=false, nodeProps: core_node6:{"core":"raceDeleteReplica_true_shard1_replica_n5","base_url":"http://127.0.0.1:42582/solr","node_name":"127.0.0.1:42582_solr","state":"down","type":"NRT"}
   [junit4]   2> 466097 INFO  (zkCallback-342-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/raceDeleteReplica_true/state.json] for collection [raceDeleteReplica_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 466097 INFO  (zkCallback-293-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/raceDeleteReplica_true/state.json] for collection [raceDeleteReplica_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 466108 INFO  (zkCallback-297-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/raceDeleteReplica_true/state.json] for collection [raceDeleteReplica_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 466108 INFO  (zkCallback-293-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/raceDeleteReplica_true/state.json] for collection [raceDeleteReplica_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 466116 INFO  (zkCallback-334-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/raceDeleteReplica_true/state.json] for collection [raceDeleteReplica_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 467003 INFO  (qtp1355906168-791) [n:127.0.0.1:45646_solr    x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=raceDeleteReplica_true, shard=shard1, thisCore=raceDeleteReplica_true_shard1_replica_n2, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:42582_solr, coreNodeName=core_node6, onlyIfActiveCheckResult=false, nodeProps: core_node6:{"core":"raceDeleteReplica_true_shard1_replica_n5","base_url":"http://127.0.0.1:42582/solr","node_name":"127.0.0.1:42582_solr","state":"recovering","type":"NRT"}
   [junit4]   2> 467003 INFO  (qtp1355906168-791) [n:127.0.0.1:45646_solr    x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node6, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
   [junit4]   2> 467003 INFO  (qtp1355906168-791) [n:127.0.0.1:45646_solr    x:raceDeleteReplica_true_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:42582_solr&onlyIfLeaderActive=true&core=raceDeleteReplica_true_shard1_replica_n2&coreNodeName=core_node6&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1022
   [junit4]   2> 467519 INFO  (recoveryExecutor-278-thread-1-processing-n:127.0.0.1:42582_solr x:raceDeleteReplica_true_shard1_replica_n5 c:raceDeleteReplica_true s:shard1 r:core_node6) [n:127.0.0.1:42582_solr c:raceDeleteReplica_true s:shard1 r:core_node6 x:raceDeleteReplica_true_shard1_replica_n5] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:45646/solr/raceDeleteReplica_true_shard1_replica_n2/] - recoveringAfterStartup=[true]
   [junit4]   2> 467592 INFO  (recoveryExecutor-278-thread-1-processing-n:127.0.0.1:42582_solr x:raceDeleteReplica_true_shard1_replica_n5 c:raceDeleteRepl

[...truncated too long message...]

gs.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null338255932
     [copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null338255932
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null338255932

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null2141967798
     [copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null2141967798
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null2141967798

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 56ms :: artifacts dl 20ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 348 minutes 44 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2535 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2535/

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.client.solrj.io.graph.GraphTest

Error Message:
Error from server at https://127.0.0.1:33551/solr: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000000

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:33551/solr: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000000
        at __randomizedtesting.SeedInfo.seed([7F9746C299892580]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.client.solrj.io.graph.GraphTest.setupCluster(GraphTest.java:62)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.client.solrj.io.stream.MathExpressionTest

Error Message:
Error from server at https://127.0.0.1:39622/solr: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000000

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:39622/solr: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000000
        at __randomizedtesting.SeedInfo.seed([7F9746C299892580]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.client.solrj.io.stream.MathExpressionTest.setupCluster(MathExpressionTest.java:62)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 15941 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.io.graph.GraphTest
   [junit4]   2> 230242 INFO  (SUITE-GraphTest-seed#[7F9746C299892580]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.graph.GraphTest_7F9746C299892580-001/init-core-data-001
   [junit4]   2> 230243 INFO  (SUITE-GraphTest-seed#[7F9746C299892580]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 230245 INFO  (SUITE-GraphTest-seed#[7F9746C299892580]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 230259 INFO  (SUITE-GraphTest-seed#[7F9746C299892580]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.graph.GraphTest_7F9746C299892580-001/tempDir-001
   [junit4]   2> 230259 INFO  (SUITE-GraphTest-seed#[7F9746C299892580]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 230287 INFO  (Thread-75) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 230287 INFO  (Thread-75) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 230304 ERROR (Thread-75) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 230396 INFO  (SUITE-GraphTest-seed#[7F9746C299892580]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:37649
   [junit4]   2> 230616 INFO  (zkConnectionManagerCallback-102-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 230806 INFO  (jetty-launcher-99-thread-2) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 230824 INFO  (jetty-launcher-99-thread-1) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 230864 INFO  (jetty-launcher-99-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 230864 INFO  (jetty-launcher-99-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 230864 INFO  (jetty-launcher-99-thread-2) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 230880 INFO  (jetty-launcher-99-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@36aba1dd{/solr,null,AVAILABLE}
   [junit4]   2> 230974 INFO  (jetty-launcher-99-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 230974 INFO  (jetty-launcher-99-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 230974 INFO  (jetty-launcher-99-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 230990 INFO  (jetty-launcher-99-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@30979b5b{/solr,null,AVAILABLE}
   [junit4]   2> 231298 INFO  (jetty-launcher-99-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@46d1af1f{SSL,[ssl, http/1.1]}{127.0.0.1:38154}
   [junit4]   2> 231298 INFO  (jetty-launcher-99-thread-2) [    ] o.e.j.s.Server Started @231667ms
   [junit4]   2> 231298 INFO  (jetty-launcher-99-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38154}
   [junit4]   2> 231299 ERROR (jetty-launcher-99-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 231299 INFO  (jetty-launcher-99-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 231299 INFO  (jetty-launcher-99-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 231299 INFO  (jetty-launcher-99-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 231299 INFO  (jetty-launcher-99-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 231299 INFO  (jetty-launcher-99-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-19T14:22:51.164Z
   [junit4]   2> 231408 INFO  (zkConnectionManagerCallback-104-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 231411 INFO  (jetty-launcher-99-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@774dfb7a{SSL,[ssl, http/1.1]}{127.0.0.1:33551}
   [junit4]   2> 231411 INFO  (jetty-launcher-99-thread-1) [    ] o.e.j.s.Server Started @231779ms
   [junit4]   2> 231411 INFO  (jetty-launcher-99-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33551}
   [junit4]   2> 231412 ERROR (jetty-launcher-99-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 231412 INFO  (jetty-launcher-99-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 231424 INFO  (jetty-launcher-99-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 231481 INFO  (jetty-launcher-99-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 231481 INFO  (jetty-launcher-99-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 231481 INFO  (jetty-launcher-99-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 231481 INFO  (jetty-launcher-99-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-19T14:22:51.346Z
   [junit4]   2> 231555 INFO  (jetty-launcher-99-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37649/solr
   [junit4]   2> 231612 INFO  (zkConnectionManagerCallback-110-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 231628 INFO  (zkConnectionManagerCallback-106-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 231646 INFO  (jetty-launcher-99-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 231684 INFO  (zkConnectionManagerCallback-112-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 231900 INFO  (jetty-launcher-99-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37649/solr
   [junit4]   2> 232038 INFO  (zkConnectionManagerCallback-116-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 232366 INFO  (zkConnectionManagerCallback-120-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 232501 WARN  (jetty-launcher-99-thread-2) [n:127.0.0.1:38154_solr    ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
   [junit4]   2> 232794 INFO  (jetty-launcher-99-thread-2) [n:127.0.0.1:38154_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 232830 INFO  (jetty-launcher-99-thread-2) [n:127.0.0.1:38154_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38154_solr
   [junit4]   2> 232904 INFO  (jetty-launcher-99-thread-2) [n:127.0.0.1:38154_solr    ] o.a.s.c.Overseer Overseer (id=72652475522482180-127.0.0.1:38154_solr-n_0000000000) starting
   [junit4]   2> 232997 WARN  (jetty-launcher-99-thread-1) [n:127.0.0.1:33551_solr    ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
   [junit4]   2> 233256 INFO  (jetty-launcher-99-thread-1) [n:127.0.0.1:33551_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 233329 INFO  (jetty-launcher-99-thread-1) [n:127.0.0.1:33551_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33551_solr
   [junit4]   2> 233439 INFO  (zkConnectionManagerCallback-127-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 233508 INFO  (zkCallback-111-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 233573 INFO  (zkCallback-119-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 233609 INFO  (jetty-launcher-99-thread-2) [n:127.0.0.1:38154_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 233756 INFO  (jetty-launcher-99-thread-2) [n:127.0.0.1:38154_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37649/solr ready
   [junit4]   2> 234137 INFO  (jetty-launcher-99-thread-2) [n:127.0.0.1:38154_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 234138 INFO  (jetty-launcher-99-thread-2) [n:127.0.0.1:38154_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38154_solr
   [junit4]   2> 234219 INFO  (zkCallback-119-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 234219 INFO  (zkCallback-111-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 234893 INFO  (zkCallback-126-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 235896 INFO  (jetty-launcher-99-thread-1) [n:127.0.0.1:33551_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33551.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b548490
   [junit4]   2> 236005 INFO  (jetty-launcher-99-thread-1) [n:127.0.0.1:33551_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33551.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b548490
   [junit4]   2> 236006 INFO  (jetty-launcher-99-thread-1) [n:127.0.0.1:33551_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33551.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b548490
   [junit4]   2> 236007 INFO  (jetty-launcher-99-thread-1) [n:127.0.0.1:33551_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.graph.GraphTest_7F9746C299892580-001/tempDir-001/node1/.
   [junit4]   2> 236244 INFO  (zkConnectionManagerCallback-132-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 236416 INFO  (jetty-launcher-99-thread-1) [n:127.0.0.1:33551_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 236417 INFO  (jetty-launcher-99-thread-1) [n:127.0.0.1:33551_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37649/solr ready
   [junit4]   2> 239168 INFO  (jetty-launcher-99-thread-2) [n:127.0.0.1:38154_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38154.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b548490
   [junit4]   2> 239300 INFO  (jetty-launcher-99-thread-2) [n:127.0.0.1:38154_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38154.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b548490
   [junit4]   2> 239301 INFO  (jetty-launcher-99-thread-2) [n:127.0.0.1:38154_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38154.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b548490
   [junit4]   2> 239302 INFO  (jetty-launcher-99-thread-2) [n:127.0.0.1:38154_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.graph.GraphTest_7F9746C299892580-001/tempDir-001/node2/.
   [junit4]   2> 239665 INFO  (zkConnectionManagerCallback-134-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 239815 INFO  (zkConnectionManagerCallback-139-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 239830 INFO  (SUITE-GraphTest-seed#[7F9746C299892580]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 239849 INFO  (SUITE-GraphTest-seed#[7F9746C299892580]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37649/solr ready
   [junit4]   2> 241953 INFO  (qtp1905425546-569) [n:127.0.0.1:33551_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 242212 INFO  (OverseerThreadFactory-372-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 243165 INFO  (OverseerStateUpdate-72652475522482180-127.0.0.1:38154_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:33551/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 243236 INFO  (OverseerStateUpdate-72652475522482180-127.0.0.1:38154_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"collection1_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:38154/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 244269 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr    x:collection1_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 244270 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr    x:collection1_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 244545 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr    x:collection1_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 245652 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 245671 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 245851 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.IndexSchema [collection1_shard2_replica_n2] Schema name=test
   [junit4]   2> 245931 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema [collection1_shard1_replica_n1] Schema name=test
   [junit4]   2> 251927 WARN  (jetty-launcher-99-thread-2-SendThread(127.0.0.1:37649)) [    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 8654ms for sessionid 0x1021d0aa3920007
   [junit4]   2> 251944 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1021d0aa3920007, likely client has closed socket
   [junit4]   2> 252035 WARN  (zkConnectionManagerCallback-127-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@38be919 name: ZooKeeperConnection Watcher:127.0.0.1:37649/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 252036 WARN  (zkConnectionManagerCallback-127-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 279266 WARN  (jetty-launcher-99-thread-2-SendThread(127.0.0.1:37649)) [    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 27321ms for sessionid 0x1021d0aa3920004
   [junit4]   2> 279267 WARN  (jetty-launcher-99-thread-1-SendThread(127.0.0.1:37649)) [    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 33378ms for sessionid 0x1021d0aa3920006
   [junit4]   2> 279267 WARN  (SUITE-GraphTest-seed#[7F9746C299892580]-worker-SendThread(127.0.0.1:37649)) [    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 27302ms for sessionid 0x1021d0aa392000a
   [junit4]   2> 279267 WARN  (jetty-launcher-99-thread-1-SendThread(127.0.0.1:37649)) [    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 27324ms for sessionid 0x1021d0aa3920008
   [junit4]   2> 279284 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1021d0aa3920008, likely client has closed socket
   [junit4]   2> 279284 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1021d0aa3920006, likely client has closed socket
   [junit4]   2> 279284 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1021d0aa392000a, likely client has closed socket
   [junit4]   2> 279284 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1021d0aa3920004, likely client has closed socket
   [junit4]   2> 279393 WARN  (zkConnectionManagerCallback-132-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2b69c64b name: ZooKeeperConnection Watcher:127.0.0.1:37649/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 279393 WARN  (zkConnectionManagerCallback-132-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 279426 WARN  (zkConnectionManagerCallback-120-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@70dba1fc name: ZooKeeperConnection Watcher:127.0.0.1:37649/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 279426 WARN  (zkConnectionManagerCallback-120-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 279511 WARN  (zkConnectionManagerCallback-139-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6373295f name: ZooKeeperConnection Watcher:127.0.0.1:37649/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 279511 WARN  (zkConnectionManagerCallback-139-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 279547 WARN  (zkConnectionManagerCallback-112-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5c759466 name: ZooKeeperConnection Watcher:127.0.0.1:37649/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 279548 WARN  (zkConnectionManagerCallback-112-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 280076 WARN  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.GeoHashField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 280115 WARN  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.LatLonType]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 280377 WARN  (jetty-launcher-99-thread-2-SendThread(127.0.0.1:37649)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x1021d0aa3920007 has expired
   [junit4]   2> 280404 WARN  (zkConnectionManagerCallback-127-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@38be919 name: ZooKeeperConnection Watcher:127.0.0.1:37649/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 280405 WARN  (zkConnectionManagerCallback-127-thread-1) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 280406 WARN  (zkConnectionManagerCallback-127-thread-1) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 280476 WARN  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 280476 WARN  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 280477 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 280491 WARN  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 280491 WARN  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 280491 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 280514 INFO  (zkConnectionManagerCallback-127-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 280515 INFO  (zkConnectionManagerCallback-127-thread-1) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 280528 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using configuration from collection collection1, trusted=true
   [junit4]   2> 280547 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n2' using configuration from collection collection1, trusted=true
   [junit4]   2> 280548 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33551.solr.core.collection1.shard1.replica_n1' (registry 'solr.core.collection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b548490
   [junit4]   2> 280548 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 280548 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [[collection1_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.graph.GraphTest_7F9746C299892580-001/tempDir-001/node1/collection1_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.graph.GraphTest_7F9746C299892580-001/tempDir-001/node1/./collection1_shard1_replica_n1/data/]
   [junit4]   2> 280563 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38154.solr.core.collection1.shard2.replica_n2' (registry 'solr.core.collection1.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b548490
   [junit4]   2> 280563 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 280563 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrCore [[collection1_shard2_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.graph.GraphTest_7F9746C299892580-001/tempDir-001/node2/collection1_shard2_replica_n2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.graph.GraphTest_7F9746C299892580-001/tempDir-001/node2/./collection1_shard2_replica_n2/data/]
   [junit4]   2> 280624 INFO  (zkConnectionManagerCallback-127-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 280707 WARN  (SUITE-GraphTest-seed#[7F9746C299892580]-worker-SendThread(127.0.0.1:37649)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x1021d0aa392000a has expired
   [junit4]   2> 280707 WARN  (zkConnectionManagerCallback-139-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6373295f name: ZooKeeperConnection Watcher:127.0.0.1:37649/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 280707 WARN  (zkConnectionManagerCallback-139-thread-1) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 280708 WARN  (zkConnectionManagerCallback-139-thread-1) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 280726 INFO  (zkConnectionManagerCallback-127-thread-1) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
   [junit4]   2> 280726 INFO  (zkConnectionManagerCallback-127-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient Connected:true
   [junit4]   2> 280857 INFO  (zkConnectionManagerCallback-139-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 280857 INFO  (zkConnectionManagerCallback-139-thread-1) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 280925 WARN  (jetty-launcher-99-thread-1-SendThread(127.0.0.1:37649)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x1021d0aa3920008 has expired
   [junit4]   2> 280925 WARN  (zkConnectionManagerCallback-132-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2b69c64b name: ZooKeeperConnection Watcher:127.0.0.1:37649/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 280925 WARN  (zkConnectionManagerCallback-132-thread-1) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 280925 WARN  (zkConnectionManagerCallback-132-thread-1) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 281021 INFO  (zkConnectionManagerCallback-139-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 281038 INFO  (zkConnectionManagerCallback-112-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 281055 INFO  (zkConnectionManagerCallback-139-thread-1) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
   [junit4]   2> 281055 INFO  (zkConnectionManagerCallback-139-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient Connected:true
   [junit4]   2> 281112 INFO  (zkCallback-111-thread-5) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 281124 INFO  (zkConnectionManagerCallback-132-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 281181 INFO  (zkConnectionManagerCallback-132-thread-1) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 281328 INFO  (zkConnectionManagerCallback-132-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 281364 INFO  (zkConnectionManagerCallback-132-thread-1) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
   [junit4]   2> 281364 INFO  (zkConnectionManagerCallback-132-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient Connected:true
   [junit4]   2> 281381 WARN  (jetty-launcher-99-thread-1-SendThread(127.0.0.1:37649)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x1021d0aa3920006 has expired
   [junit4]   2> 281382 WARN  (zkConnectionManagerCallback-120-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@70dba1fc name: ZooKeeperConnection Watcher:127.0.0.1:37649/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 281382 WARN  (zkConnectionManagerCallback-120-thread-1) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 281382 WARN  (zkConnectionManagerCallback-120-thread-1) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 281417 INFO  (zkConnectionManagerCallback-120-thread-1-EventThread) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 281461 INFO  (zkConnectionManagerCallback-120-thread-1) [    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 281461 INFO  (zkConnectionManagerCallback-120-thread-1) [    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 281637 INFO  (zkConnectionManagerCallback-120-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 281658 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 281658 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 281659 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 281660 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 281762 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 281762 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 281764 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 281764 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 281802 INFO  (zkConnectionManagerCallback-120-thread-1) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33551_solr
   [junit4]   2> 281873 INFO  (zkCallback-131-thread-4) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 281880 INFO  (zkCallback-126-thread-4) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 281874 INFO  (zkCallback-111-thread-6) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 281873 INFO  (zkCallback-138-thread-4) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 282076 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@356c3864[collection1_shard1_replica_n1] main]
   [junit4]   2> 282075 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@6e6f81ff[collection1_shard2_replica_n2] main]
   [junit4]   2> 281982 INFO  (zkCallback-119-thread-4) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 281982 INFO  (zkConnectionManagerCallback-120-thread-1) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
   [junit4]   2> 282096 INFO  (zkConnectionManagerCallback-120-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient Connected:true
   [junit4]   2> 282099 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 282117 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 282133 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 282133 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 282153 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 282155 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 282155 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600902735614443520
   [junit4]   2> 282161 INFO  (searcherExecutor-378-thread-1-processing-n:127.0.0.1:38154_solr x:collection1_shard2_replica_n2 c:collection1 s:shard2 r:core_node4) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrCore [collection1_shard2_replica_n2] Registered new searcher Searcher@6e6f81ff[collection1_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 282169 INFO  (searcherExecutor-377-thread-1-processing-n:127.0.0.1:33551_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [collection1_shard1_replica_n1] Registered new searcher Searcher@356c3864[collection1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 282169 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600902735629123584
   [junit4]   2> 282575 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node4=0}, version=0}
   [junit4]   2> 282610 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node3=0}, version=0}
   [junit4]   2> 282662 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 282662 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 282663 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:33551/solr/collection1_shard1_replica_n1/
   [junit4]   2> 282684 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 282684 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 282684 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:38154/solr/collection1_shard2_replica_n2/
   [junit4]   2> 282684 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 282684 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 282701 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SyncStrategy https://127.0.0.1:38154/solr/collection1_shard2_replica_n2/ has no replicas
   [junit4]   2> 282702 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 282718 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:33551/solr/collection1_shard1_replica_n1/ has no replicas
   [junit4]   2> 282718 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 282866 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:38154/solr/collection1_shard2_replica_n2/ shard2
   [junit4]   2> 282883 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:33551/solr/collection1_shard1_replica_n1/ shard1
   [junit4]   2> 283084 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 283128 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 283167 INFO  (qtp1428074433-561) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=38622
   [junit4]   2> 283185 INFO  (qtp1905425546-568) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=38916
   [junit4]   2> 283295 INFO  (OverseerThreadFactory-372-thread-1) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 283370 INFO  (zkCallback-111-thread-6) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 283379 INFO  (zkCallback-119-thread-4) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 285119 INFO  (OverseerCollectionConfigSetProcessor-72652475522482180-127.0.0.1:38154_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 422047 ERROR (qtp1905425546-569) [n:127.0.0.1:33551_solr    ] o.a.s.h.RequestHandlerBase org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000000
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:114)
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
   [junit4]   2> at org.apache.solr.cloud.OverseerTaskQueue.offer(OverseerTaskQueue.java:210)
   [junit4]   2> at org.apache.solr.handler.admin.CollectionsHandler.sendToOCPQueue(CollectionsHandler.java:348)
   [junit4]   2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:258)
   [junit4]   2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:232)
   [junit4]   2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:324)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
   [junit4]   2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:724)
   [junit4]   2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
   [junit4]   2> at org.eclipse.jetty.server.Server.handle(Server.java:531)
   [junit4]   2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
   [junit4]   2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
   [junit4]   2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:291)
   [junit4]   2> at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:151)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
   [junit4]   2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
   [junit4]   2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>
   [junit4]   2> 422047 INFO  (qtp1905425546-569) [n:127.0.0.1:33551_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=500 QTime=180094
   [junit4]   2> 422047 ERROR (qtp1905425546-569) [n:127.0.0.1:33551_solr    ] o.a.s.s.HttpSolrCall null:org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000000
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:114)
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
   [junit4]   2> at org.apache.solr.cloud.OverseerTaskQueue.offer(OverseerTaskQueue.java:210)
   [junit4]   2> at org.apache.solr.handler.admin.CollectionsHandler.sendToOCPQueue(CollectionsHandler.java:348)
   [junit4]   2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:258)
   [junit4]   2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:232)
   [junit4]   2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:324)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
   [junit4]   2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:724)
   [junit4]   2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
   [junit4]   2> at org.eclipse.jetty.server.Server.handle(Server.java:531)
   [junit4]   2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
   [junit4]   2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
   [junit4]   2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:291)
   [junit4]   2> at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:151)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
   [junit4]   2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
   [junit4]   2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>
   [junit4]   2> 422221 INFO  (jetty-closer-100-thread-2) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@46d1af1f{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 422223 INFO  (jetty-closer-100-thread-2) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1121472569
   [junit4]   2> 422223 INFO  (jetty-closer-100-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 422223 INFO  (jetty-closer-100-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5b31f92d: rootName = solr_38154, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@3f60d990
   [junit4]   2> 422244 INFO  (jetty-closer-100-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 422244 INFO  (jetty-closer-100-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@146f694d: rootName = solr_38154, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2d43d0e6
   [junit4]   2> 422279 INFO  (jetty-closer-100-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 422279 INFO  (jetty-closer-100-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@35634c5: rootName = solr_38154, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@79022706
   [junit4]   2> 422282 INFO  (jetty-closer-100-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@774dfb7a{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 422295 INFO  (jetty-closer-100-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=277590539
   [junit4]   2> 422295 INFO  (jetty-closer-100-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 422295 INFO  (jetty-closer-100-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1f1303c4: rootName = solr_33551, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@4428df10
   [junit4]   2> 422317 INFO  (jetty-closer-100-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 422317 INFO  (jetty-closer-100-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@cafcefe: rootName = solr_33551, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2d43d0e6
   [junit4]   2> 422335 INFO  (jetty-closer-100-thread-2) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:38154_solr
   [junit4]   2> 422336 INFO  (jetty-closer-100-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 422353 INFO  (jetty-closer-100-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 422354 INFO  (jetty-closer-100-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4d7f40de: rootName = solr_33551, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@79022706
   [junit4]   2> 422371 INFO  (zkCallback-111-thread-7) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 422505 INFO  (zkCallback-126-thread-5) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 422513 INFO  (zkCallback-119-thread-7) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 422517 INFO  (zkCallback-131-thread-5) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 422517 INFO  (zkCallback-111-thread-8) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 422659 INFO  (coreCloseExecutor-387-thread-1) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrCore [collection1_shard2_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@d4b8974
   [junit4]   2> 422659 INFO  (coreCloseExecutor-387-thread-1) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard2.replica_n2, tag=d4b8974
   [junit4]   2> 422660 INFO  (coreCloseExecutor-387-thread-1) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5ba99ca4: rootName = solr_38154, domain = solr.core.collection1.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.collection1.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@7cbc88c2
   [junit4]   2> 422675 INFO  (zkCallback-119-thread-6) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 422696 INFO  (jetty-closer-100-thread-1) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:33551_solr
   [junit4]   2> 422705 INFO  (jetty-closer-100-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 422706 INFO  (zkCallback-126-thread-5) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 422706 INFO  (zkCallback-119-thread-6) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 422719 INFO  (coreCloseExecutor-387-thread-1) [n:127.0.0.1:38154_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard2.leader, tag=d4b8974
   [junit4]   2> 422739 INFO  (coreCloseExecutor-388-thread-1) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [collection1_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@3138b9ad
   [junit4]   2> 422739 INFO  (coreCloseExecutor-388-thread-1) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_n1, tag=3138b9ad
   [junit4]   2> 422739 INFO  (coreCloseExecutor-388-thread-1) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@aa6bbd5: rootName = solr_33551, domain = solr.core.collection1.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection1.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@79105e27
   [junit4]   2> 422748 INFO  (zkCallback-111-thread-9) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 422753 INFO  (zkCallback-131-thread-5) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 422901 INFO  (coreCloseExecutor-388-thread-1) [n:127.0.0.1:33551_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=3138b9ad
   [junit4]   2> 422956 INFO  (zkCallback-119-thread-6) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
   [junit4]   2> 422968 INFO  (zkCallback-111-thread-9) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
   [junit4]   2> 423047 INFO  (jetty-closer-100-thread-2) [    ] o.a.s.c.Overseer Overseer (id=72652475522482180-127.0.0.1:38154_solr-n_0000000000) closing
   [junit4]   2> 423047 INFO  (OverseerStateUpdate-72652475522482180-127.0.0.1:38154_solr-n_0000000000) [    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:38154_solr
   [junit4]   2> 423090 WARN  (OverseerAutoScalingTriggerThread-72652475522482180-127.0.0.1:38154_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 423156 INFO  (jetty-closer-100-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@30979b5b{/solr,null,UNAVAILABLE}
   [junit4]   2> 423156 INFO  (jetty-closer-100-thread-1) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 423230 INFO  (jetty-closer-100-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@36aba1dd{/solr,null,UNAVAILABLE}
   [junit4]   2> 423230 INFO  (jetty-closer-100-thread-2) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 423249 ERROR (SUITE-GraphTest-seed#[7F9746C299892580]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 423249 INFO  (SUITE-GraphTest-seed#[7F9746C299892580]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37649 37649
   [junit4]   2> 423558 INFO  (Thread-75) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37649 37649
   [junit4]   2> 423558 WARN  (Thread-75) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 9 /solr/aliases.json
   [junit4]   2> 3 /solr/security.json
   [junit4]   2> 2 /solr/collections/collection1/terms/shard2
   [junit4]   2> 2 /solr/configs/conf
   [junit4]   2> 2 /solr/collections/collection1/terms/shard1
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 9 /solr/clusterstate.json
   [junit4]   2> 9 /solr/clusterprops.json
   [junit4]   2> 2 /solr/collections/collection1/state.json
   [junit4]   2> 2 /solr/overseer_elect/election/72652475522482180-127.0.0.1:38154_solr-n_0000000000
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 9 /solr/live_nodes
   [junit4]   2> 9 /solr/collections
   [junit4]   2>
   [junit4]   2> 423559 INFO  (SUITE-GraphTest-seed#[7F9746C299892580]-worker) [    ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
   [junit4]   2> May 19, 2018 2:26:03 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene70, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@25eee26c), locale=be-BY, timezone=Africa/Maseru
   [junit4]   2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=219140080,total=392691712
   [junit4]   2> NOTE: All tests run in this JVM: [TestBatchUpdate, HyperbolicSineEvaluatorTest, HttpSolrClientBuilderTest, LessThanEqualToEvaluatorTest, NoOpResponseParserTest, StreamExpressionToExplanationTest, TestSuggesterResponse, SolrParamTest, SineEvaluatorTest, TangentEvaluatorTest, GreaterThanEqualToEvaluatorTest, TestCoreAdmin, NamedListTest, JettyWebappTest, HttpClientUtilTest, PowerEvaluatorTest, TestHash, SolrExampleBinaryTest, CloudSolrClientBuilderTest, TestCloudSolrClientConnections, GraphTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=GraphTest -Dtests.seed=7F9746C299892580 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=be-BY -Dtests.timezone=Africa/Maseru -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J2 | GraphTest (suite) <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:33551/solr: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000000
   [junit4]    > at __randomizedtesting.SeedInfo.seed([7F9746C299892580]:0)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
   [junit4]    > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
   [junit4]    > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
   [junit4]    > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
   [junit4]    > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
   [junit4]    > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
   [junit4]    > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
   [junit4]    > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
   [junit4]    > at org.apache.solr.client.solrj.io.graph.GraphTest.setupCluster(GraphTest.java:62)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4] Completed [66/152 (1!)] on J2 in 194.35s, 0 tests, 1 error <<< FAILURES!

[...truncated 94 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.io.stream.MathExpressionTest
   [junit4]   2> 389192 INFO  (SUITE-MathExpressionTest-seed#[7F9746C299892580]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.io.stream.MathExpressionTest_7F9746C299892580-001/init-core-data-001
   [junit4]   2> 389192 INFO  (SUITE-MathExpressionTest-seed#[7F9746C299892580]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 389193 INFO  (SUITE-MathExpressionTest-seed#[7F9746C299892580]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 389195 INFO  (SUITE-MathExpressionTest-seed#[7F9746C299892580]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.io.stream.MathExpressionTest_7F9746C299892580-001/tempDir-001
   [junit4]   2> 389195 INFO  (SUITE-MathExpressionTest-seed#[7F9746C299892580]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 389212 INFO  (Thread-433) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 389212 INFO  (Thread-433) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 389229 ERROR (Thread-433) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 389312 INFO  (SUITE-MathExpressionTest-seed#[7F9746C299892580]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:39271
   [junit4]   2> 389496 INFO  (zkConnectionManagerCallback-681-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 389538 INFO  (jetty-launcher-678-thread-2) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 389539 INFO  (jetty-launcher-678-thread-1) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 389590 INFO  (jetty-launcher-678-thread-4) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 389646 INFO  (jetty-launcher-678-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 389647 INFO  (jetty-launcher-678-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 389647 INFO  (jetty-launcher-678-thread-4) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 389648 INFO  (jetty-launcher-678-thread-3) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 389651 INFO  (jetty-launcher-678-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@790c786b{/solr,null,AVAILABLE}
   [junit4]   2> 389725 INFO  (jetty-launcher-678-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 389725 INFO  (jetty-launcher-678-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 389725 INFO  (jetty-launcher-678-thread-3) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 389725 INFO  (jetty-launcher-678-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@b20b2db{/solr,null,AVAILABLE}
   [junit4]   2> 389727 INFO  (jetty-launcher-678-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2c2b5d0b{SSL,[ssl, http/1.1]}{127.0.0.1:41156}
   [junit4]   2> 389727 INFO  (jetty-launcher-678-thread-3) [    ] o.e.j.s.Server Started @390103ms
   [junit4]   2> 389727 INFO  (jetty-launcher-678-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41156}
   [junit4]   2> 389727 ERROR (jetty-launcher-678-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 389727 INFO  (jetty-launcher-678-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 389727 INFO  (jetty-launcher-678-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 389727 INFO  (jetty-launcher-678-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 389727 INFO  (jetty-launcher-678-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 389727 INFO  (jetty-launcher-678-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-19T14:25:29.563Z
   [junit4]   2> 389747 INFO  (jetty-launcher-678-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 389747 INFO  (jetty-launcher-678-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 389747 INFO  (jetty-launcher-678-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 389747 INFO  (jetty-launcher-678-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e37b8d{/solr,null,AVAILABLE}
   [junit4]   2> 389748 INFO  (jetty-launcher-678-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@428c8d{SSL,[ssl, http/1.1]}{127.0.0.1:35137}
   [junit4]   2> 389748 INFO  (jetty-launcher-678-thread-1) [    ] o.e.j.s.Server Started @390125ms
   [junit4]   2> 389748 INFO  (jetty-launcher-678-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35137}
   [junit4]   2> 389748 ERROR (jetty-launcher-678-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 389748 INFO  (jetty-launcher-678-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 389748 INFO  (jetty-launcher-678-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 389748 INFO  (jetty-launcher-678-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 389748 INFO  (jetty-launcher-678-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 389748 INFO  (jetty-launcher-678-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-19T14:25:29.584Z
   [junit4]   2> 389814 INFO  (jetty-launcher-678-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@317a523{SSL,[ssl, http/1.1]}{127.0.0.1:35667}
   [junit4]   2> 389814 INFO  (jetty-launcher-678-thread-4) [    ] o.e.j.s.Server Started @390191ms
   [junit4]   2> 389814 INFO  (jetty-launcher-678-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35667}
   [junit4]   2> 389814 ERROR (jetty-launcher-678-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 389814 INFO  (jetty-launcher-678-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 389814 INFO  (jetty-launcher-678-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 389815 INFO  (jetty-launcher-678-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 389815 INFO  (jetty-launcher-678-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 389815 INFO  (jetty-launcher-678-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-19T14:25:29.651Z
   [junit4]   2> 389829 INFO  (jetty-launcher-678-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 389830 INFO  (jetty-launcher-678-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 389830 INFO  (jetty-launcher-678-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 389831 INFO  (jetty-launcher-678-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3e7116a2{/solr,null,AVAILABLE}
   [junit4]   2> 389831 INFO  (jetty-launcher-678-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7afe7577{SSL,[ssl, http/1.1]}{127.0.0.1:39622}
   [junit4]   2> 389831 INFO  (jetty-launcher-678-thread-2) [    ] o.e.j.s.Server Started @390208ms
   [junit4]   2> 389831 INFO  (jetty-launcher-678-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39622}
   [junit4]   2> 389832 ERROR (jetty-launcher-678-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 389832 INFO  (jetty-launcher-678-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 389832 INFO  (jetty-launcher-678-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 389832 INFO  (jetty-launcher-678-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 389832 INFO  (jetty-launcher-678-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 389832 INFO  (jetty-launcher-678-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-05-19T14:25:29.668Z
   [junit4]   2> 389863 INFO  (zkConnectionManagerCallback-689-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 389863 INFO  (jetty-launcher-678-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 389865 INFO  (zkConnectionManagerCallback-685-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 389865 INFO  (jetty-launcher-678-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 389878 INFO  (zkConnectionManagerCallback-687-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 389879 INFO  (jetty-launcher-678-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 389899 INFO  (zkConnectionManagerCallback-683-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 389907 INFO  (jetty-launcher-678-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 390007 INFO  (jetty-launcher-678-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39271/solr
   [junit4]   2> 390089 INFO  (zkConnectionManagerCallback-693-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 390167 INFO  (jetty-launcher-678-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39271/solr
   [junit4]   2> 390226 INFO  (jetty-launcher-678-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39271/solr
   [junit4]   2> 390268 INFO  (zkConnectionManagerCallback-703-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 390285 INFO  (zkConnectionManagerCallback-697-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 390403 INFO  (zkConnectionManagerCallback-699-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 390485 INFO  (jetty-launcher-678-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39271/solr
   [junit4]   2> 390557 INFO  (zkConnectionManagerCallback-711-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected

[...truncated too long message...]

ings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null1432379795
     [copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null1432379795
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null1432379795

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null655049244
     [copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null655049244
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null655049244

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 43ms :: artifacts dl 1ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 273 minutes 2 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2536 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2536/

4 tests failed.
FAILED:  org.apache.solr.cloud.TestCryptoKeys.test

Error Message:
Error from server at <a href="https://127.0.0.1:33174:">https://127.0.0.1:33174: At least one of the node(s) specified [127.0.0.1:38357_] are not currently active in [], no action taken.

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at <a href="https://127.0.0.1:33174:">https://127.0.0.1:33174: At least one of the node(s) specified [127.0.0.1:38357_] are not currently active in [], no action taken.
        at __randomizedtesting.SeedInfo.seed([949C4C95DF71A91F:1CC8734F718DC4E7]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:425)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:341)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestCryptoKeys

Error Message:
92 threads leaked from SUITE scope at org.apache.solr.cloud.TestCryptoKeys:     1) Thread[id=9364, name=qtp798825832-9364, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)         at java.lang.Thread.run(Thread.java:748)    2) Thread[id=9375, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)    3) Thread[id=9340, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)    4) Thread[id=9285, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)    5) Thread[id=9367, name=qtp798825832-9367, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)         at java.lang.Thread.run(Thread.java:748)    6) Thread[id=9328, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)    7) Thread[id=9381, name=zkConnectionManagerCallback-3309-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    8) Thread[id=9253, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)    9) Thread[id=9299, name=zkConnectionManagerCallback-3284-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   10) Thread[id=9319, name=qtp26341971-9319, state=RUNNABLE, group=TGRP-TestCryptoKeys]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)         at java.lang.Thread.run(Thread.java:748)   11) Thread[id=9397, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   12) Thread[id=9400, name=zkConnectionManagerCallback-3316-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   13) Thread[id=9361, name=qtp798825832-9361, state=RUNNABLE, group=TGRP-TestCryptoKeys]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)         at org.eclipse.jetty.io.ManagedSelector$$Lambda$266/4850828.run(Unknown Source)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)         at java.lang.Thread.run(Thread.java:748)   14) Thread[id=9348, name=searcherExecutor-1629-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   15) Thread[id=9301, name=Scheduler-479120694, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   16) Thread[id=9297, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)   17) Thread[id=9344, name=Scheduler-198885779, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   18) Thread[id=9258, name=Thread-612, state=WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1743)   19) Thread[id=9272, name=qtp996720602-9272, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)         at java.lang.Thread.run(Thread.java:748)   20) Thread[id=9248, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   21) Thread[id=9362, name=qtp798825832-9362, state=RUNNABLE, group=TGRP-TestCryptoKeys]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)         at org.eclipse.jetty.io.ManagedSelector$$Lambda$266/4850828.run(Unknown Source)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)         at java.lang.Thread.run(Thread.java:748)   22) Thread[id=9373, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   23) Thread[id=9365, name=qtp798825832-9365, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)         at java.lang.Thread.run(Thread.java:748)   24) Thread[id=9380, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)   25) Thread[id=9320, name=qtp26341971-9320, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)         at java.lang.Thread.run(Thread.java:748)   26) Thread[id=9273, name=qtp996720602-9273-acceptor-0@6a16b359-ServerConnector@64d1506b{SSL,[ssl, http/1.1]}{127.0.0.1:44582}, state=RUNNABLE, group=TGRP-TestCryptoKeys]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)         at java.lang.Thread.run(Thread.java:748)   27) Thread[id=9341, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)   28) Thread[id=9236, name=qtp1267771131-9236, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)         at java.lang.Thread.run(Thread.java:748)   29) Thread[id=9241, name=qtp1267771131-9241, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)         at java.lang.Thread.run(Thread.java:748)   30) Thread[id=9343, name=zkConnectionManagerCallback-3300-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   31) Thread[id=9255, name=zkConnectionManagerCallback-3262-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   32) Thread[id=9271, name=qtp996720602-9271, state=RUNNABLE, group=TGRP-TestCryptoKeys]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)         at org.eclipse.jetty.io.ManagedSelector$$Lambda$266/4850828.run(Unknown Source)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)         at java.lang.Thread.run(Thread.java:748)   33) Thread[id=9313, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   34) Thread[id=9369, name=Scheduler-294960700, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   35) Thread[id=9263, name=zkConnectionManagerCallback-3269-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   36) Thread[id=9283, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   37) Thread[id=9254, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)   38) Thread[id=9289, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)   39) Thread[id=9342, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)   40) Thread[id=9399, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)   41) Thread[id=9243, name=Scheduler-1728744197, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   42) Thread[id=9291, name=zkConnectionManagerCallback-3277-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   43) Thread[id=9296, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   44) Thread[id=9242, name=qtp1267771131-9242, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)         at java.lang.Thread.run(Thread.java:748)   45) Thread[id=9279, name=Scheduler-2131062326, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   46) Thread[id=9355, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   47) Thread[id=9235, name=qtp1267771131-9235, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)         at java.lang.Thread.run(Thread.java:748)   48) Thread[id=9323, name=Scheduler-1689285055, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   49) Thread[id=9411, name=OverseerCollectionConfigSetProcessor-72654951734312971-127.0.0.1:36882_-n_0000000001, state=TIMED_WAITING, group=Overseer collection creation process.]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)         at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)         at org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:392)         at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:152)         at java.lang.Thread.run(Thread.java:748)   50) Thread[id=9270, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   51) Thread[id=9329, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   52) Thread[id=9334, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)   53) Thread[id=9264, name=Scheduler-409575461, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   54) Thread[id=9412, name=OverseerAutoScalingTriggerThread-72654951734312971-127.0.0.1:36882_-n_0000000001, state=WAITING, group=Overseer autoscaling triggers]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1409)         at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1213)         at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)         at org.apache.solr.common.cloud.SolrZkClient$$Lambda$270/1880615767.execute(Unknown Source)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)         at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)         at org.apache.solr.client.solrj.impl.ZkDistribStateManager.getAutoScalingConfig(ZkDistribStateManager.java:176)         at org.apache.solr.client.solrj.cloud.DistribStateManager.getAutoScalingConfig(DistribStateManager.java:83)         at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:131)         at java.lang.Thread.run(Thread.java:748)   55) Thread[id=9261, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)   56) Thread[id=9374, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   57) Thread[id=9260, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   58) Thread[id=9395, name=Thread-639, state=WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1743)   59) Thread[id=9239, name=qtp1267771131-9239, state=RUNNABLE, group=TGRP-TestCryptoKeys]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)         at java.lang.Thread.run(Thread.java:748)   60) Thread[id=9274, name=qtp996720602-9274, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)         at java.lang.Thread.run(Thread.java:748)   61) Thread[id=9275, name=qtp996720602-9275, state=RUNNABLE, group=TGRP-TestCryptoKeys]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)         at java.lang.Thread.run(Thread.java:748)   62) Thread[id=9284, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   63) Thread[id=9238, name=qtp1267771131-9238, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)         at java.lang.Thread.run(Thread.java:748)   64) Thread[id=9262, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)   65) Thread[id=9247, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   66) Thread[id=9249, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   67) Thread[id=9240, name=qtp1267771131-9240, state=RUNNABLE, group=TGRP-TestCryptoKeys]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)         at java.lang.Thread.run(Thread.java:748)   68) Thread[id=9298, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)   69) Thread[id=9276, name=qtp996720602-9276, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)         at java.lang.Thread.run(Thread.java:748)   70) Thread[id=9277, name=qtp996720602-9277, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)         at java.lang.Thread.run(Thread.java:748)   71) Thread[id=9294, name=Thread-621, state=WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1743)   72) Thread[id=9303, name=searcherExecutor-1618-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   73) Thread[id=9267, name=searcherExecutor-1607-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   74) Thread[id=9322, name=qtp26341971-9322, state=RUNNABLE, group=TGRP-TestCryptoKeys]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)         at java.lang.Thread.run(Thread.java:748)   75) Thread[id=9338, name=Thread-630, state=WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1743)   76) Thread[id=9379, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)   77) Thread[id=9333, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:230)         at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1249)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1173)   78) Thread[id=9327, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   79) Thread[id=9366, name=qtp798825832-9366, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)         at java.lang.Thread.run(Thread.java:748)   80) Thread[id=9278, name=qtp996720602-9278, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)         at java.lang.Thread.run(Thread.java:748)   81) Thread[id=9317, name=qtp26341971-9317-acceptor-0@21929e9-ServerConnector@42291d2f{SSL,[ssl, http/1.1]}{127.0.0.1:33174}, state=RUNNABLE, group=TGRP-TestCryptoKeys]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)         at java.lang.Thread.run(Thread.java:748)   82) Thread[id=9237, name=qtp1267771131-9237-acceptor-0@420ac588-ServerConnector@4fbba792{SSL,[ssl, http/1.1]}{127.0.0.1:36882}, state=RUNNABLE, group=TGRP-TestCryptoKeys]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)         at java.lang.Thread.run(Thread.java:748)   83) Thread[id=9316, name=qtp26341971-9316, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)         at java.lang.Thread.run(Thread.java:748)   84) Thread[id=9363, name=qtp798825832-9363-acceptor-0@a0caadb-ServerConnector@23dc0a1{SSL,[ssl, http/1.1]}{127.0.0.1:38357}, state=RUNNABLE, group=TGRP-TestCryptoKeys]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)         at java.lang.Thread.run(Thread.java:748)   85) Thread[id=9321, name=qtp26341971-9321, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)         at java.lang.Thread.run(Thread.java:748)   86) Thread[id=9368, name=qtp798825832-9368, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)         at java.lang.Thread.run(Thread.java:748)   87) Thread[id=9290, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)   88) Thread[id=9398, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)   89) Thread[id=9315, name=qtp26341971-9315, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)         at java.lang.Thread.run(Thread.java:748)   90) Thread[id=9318, name=qtp26341971-9318, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)         at java.lang.Thread.run(Thread.java:748)   91) Thread[id=9410, name=OverseerStateUpdate-72654951734312971-127.0.0.1:36882_-n_0000000001, state=TIMED_WAITING, group=Overseer state updater.]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)         at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)         at org.apache.solr.cloud.Overseer$ClusterStateUpdater.amILeader(Overseer.java:417)         at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:145)         at java.lang.Thread.run(Thread.java:748)   92) Thread[id=9335, name=zkConnectionManagerCallback-3293-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 92 threads leaked from SUITE scope at org.apache.solr.cloud.TestCryptoKeys:
   1) Thread[id=9364, name=qtp798825832-9364, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
        at java.lang.Thread.run(Thread.java:748)
   2) Thread[id=9375, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
   3) Thread[id=9340, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
   4) Thread[id=9285, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
   5) Thread[id=9367, name=qtp798825832-9367, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
        at java.lang.Thread.run(Thread.java:748)
   6) Thread[id=9328, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
   7) Thread[id=9381, name=zkConnectionManagerCallback-3309-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   8) Thread[id=9253, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)
   9) Thread[id=9299, name=zkConnectionManagerCallback-3284-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  10) Thread[id=9319, name=qtp26341971-9319, state=RUNNABLE, group=TGRP-TestCryptoKeys]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
        at java.lang.Thread.run(Thread.java:748)
  11) Thread[id=9397, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  12) Thread[id=9400, name=zkConnectionManagerCallback-3316-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  13) Thread[id=9361, name=qtp798825832-9361, state=RUNNABLE, group=TGRP-TestCryptoKeys]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at org.eclipse.jetty.io.ManagedSelector$$Lambda$266/4850828.run(Unknown Source)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
        at java.lang.Thread.run(Thread.java:748)
  14) Thread[id=9348, name=searcherExecutor-1629-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  15) Thread[id=9301, name=Scheduler-479120694, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  16) Thread[id=9297, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)
  17) Thread[id=9344, name=Scheduler-198885779, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  18) Thread[id=9258, name=Thread-612, state=WAITING, group=TGRP-TestCryptoKeys]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1743)
  19) Thread[id=9272, name=qtp996720602-9272, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
        at java.lang.Thread.run(Thread.java:748)
  20) Thread[id=9248, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  21) Thread[id=9362, name=qtp798825832-9362, state=RUNNABLE, group=TGRP-TestCryptoKeys]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at org.eclipse.jetty.io.ManagedSelector$$Lambda$266/4850828.run(Unknown Source)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
        at java.lang.Thread.run(Thread.java:748)
  22) Thread[id=9373, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  23) Thread[id=9365, name=qtp798825832-9365, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
        at java.lang.Thread.run(Thread.java:748)
  24) Thread[id=9380, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
  25) Thread[id=9320, name=qtp26341971-9320, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
        at java.lang.Thread.run(Thread.java:748)
  26) Thread[id=9273, name=qtp996720602-9273-acceptor-0@6a16b359-ServerConnector@64d1506b{SSL,[ssl, http/1.1]}{127.0.0.1:44582}, state=RUNNABLE, group=TGRP-TestCryptoKeys]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
        at java.lang.Thread.run(Thread.java:748)
  27) Thread[id=9341, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
  28) Thread[id=9236, name=qtp1267771131-9236, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
        at java.lang.Thread.run(Thread.java:748)
  29) Thread[id=9241, name=qtp1267771131-9241, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
        at java.lang.Thread.run(Thread.java:748)
  30) Thread[id=9343, name=zkConnectionManagerCallback-3300-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  31) Thread[id=9255, name=zkConnectionManagerCallback-3262-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  32) Thread[id=9271, name=qtp996720602-9271, state=RUNNABLE, group=TGRP-TestCryptoKeys]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at org.eclipse.jetty.io.ManagedSelector$$Lambda$266/4850828.run(Unknown Source)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
        at java.lang.Thread.run(Thread.java:748)
  33) Thread[id=9313, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(T

[...truncated too long message...]

ttings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null582407327
     [copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null582407327
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null582407327

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null991243315
     [copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null991243315
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null991243315

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 43ms :: artifacts dl 1ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 245 minutes 54 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]