[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1609 - Unstable!

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1609 - Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1609/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.DeleteShardTest.test

Error Message:
Could not find collection : deleteShard

Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : deleteShard
        at __randomizedtesting.SeedInfo.seed([B018BFF602F6BE9C:384C802CAC0AD364]:0)
        at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:118)
        at org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:247)
        at org.apache.solr.cloud.DeleteShardTest.test(DeleteShardTest.java:59)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12926 lines...]
   [junit4] Suite: org.apache.solr.cloud.DeleteShardTest
   [junit4]   2> 2340180 INFO  (SUITE-DeleteShardTest-seed#[B018BFF602F6BE9C]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest_B018BFF602F6BE9C-001/init-core-data-001
   [junit4]   2> 2340181 WARN  (SUITE-DeleteShardTest-seed#[B018BFF602F6BE9C]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=7 numCloses=7
   [junit4]   2> 2340181 INFO  (SUITE-DeleteShardTest-seed#[B018BFF602F6BE9C]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 2340182 INFO  (SUITE-DeleteShardTest-seed#[B018BFF602F6BE9C]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2340182 INFO  (SUITE-DeleteShardTest-seed#[B018BFF602F6BE9C]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest_B018BFF602F6BE9C-001/tempDir-001
   [junit4]   2> 2340183 INFO  (SUITE-DeleteShardTest-seed#[B018BFF602F6BE9C]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2340183 INFO  (Thread-8824) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2340183 INFO  (Thread-8824) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2340184 ERROR (Thread-8824) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2340283 INFO  (SUITE-DeleteShardTest-seed#[B018BFF602F6BE9C]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:46016
   [junit4]   2> 2340285 INFO  (zkConnectionManagerCallback-4157-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2340288 INFO  (jetty-launcher-4154-thread-1) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 2340288 INFO  (jetty-launcher-4154-thread-2) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 2340291 INFO  (jetty-launcher-4154-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@333c4225{/solr,null,AVAILABLE}
   [junit4]   2> 2340291 INFO  (jetty-launcher-4154-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5179fecb{HTTP/1.1,[http/1.1]}{127.0.0.1:42028}
   [junit4]   2> 2340291 INFO  (jetty-launcher-4154-thread-1) [    ] o.e.j.s.Server Started @2344179ms
   [junit4]   2> 2340291 INFO  (jetty-launcher-4154-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42028}
   [junit4]   2> 2340291 INFO  (jetty-launcher-4154-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@72e834c9{/solr,null,AVAILABLE}
   [junit4]   2> 2340292 ERROR (jetty-launcher-4154-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2340292 INFO  (jetty-launcher-4154-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@45e331b1{HTTP/1.1,[http/1.1]}{127.0.0.1:36783}
   [junit4]   2> 2340292 INFO  (jetty-launcher-4154-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 2340292 INFO  (jetty-launcher-4154-thread-2) [    ] o.e.j.s.Server Started @2344179ms
   [junit4]   2> 2340292 INFO  (jetty-launcher-4154-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2340292 INFO  (jetty-launcher-4154-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36783}
   [junit4]   2> 2340292 INFO  (jetty-launcher-4154-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2340292 INFO  (jetty-launcher-4154-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-06T11:47:07.980Z
   [junit4]   2> 2340292 ERROR (jetty-launcher-4154-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2340292 INFO  (jetty-launcher-4154-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 2340292 INFO  (jetty-launcher-4154-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2340292 INFO  (jetty-launcher-4154-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2340292 INFO  (jetty-launcher-4154-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-06T11:47:07.980Z
   [junit4]   2> 2340294 INFO  (zkConnectionManagerCallback-4161-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2340294 INFO  (zkConnectionManagerCallback-4159-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2340294 INFO  (jetty-launcher-4154-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2340295 INFO  (jetty-launcher-4154-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2340300 INFO  (jetty-launcher-4154-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46016/solr
   [junit4]   2> 2340301 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x160cb4bc1e00002, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 2340301 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x160cb4bc1e00001, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 2340301 INFO  (jetty-launcher-4154-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46016/solr
   [junit4]   2> 2340301 INFO  (zkConnectionManagerCallback-4165-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2340306 INFO  (zkConnectionManagerCallback-4169-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2340307 INFO  (zkConnectionManagerCallback-4171-thread-1-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2340309 INFO  (zkConnectionManagerCallback-4173-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2340343 INFO  (jetty-launcher-4154-thread-2) [n:127.0.0.1:36783_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2340344 INFO  (jetty-launcher-4154-thread-2) [n:127.0.0.1:36783_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36783_solr
   [junit4]   2> 2340345 INFO  (jetty-launcher-4154-thread-2) [n:127.0.0.1:36783_solr    ] o.a.s.c.Overseer Overseer (id=99302718037819398-127.0.0.1:36783_solr-n_0000000000) starting
   [junit4]   2> 2340354 INFO  (jetty-launcher-4154-thread-2) [n:127.0.0.1:36783_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36783_solr
   [junit4]   2> 2340372 INFO  (zkCallback-4172-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2340466 INFO  (jetty-launcher-4154-thread-2) [n:127.0.0.1:36783_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36783.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@639a08ba
   [junit4]   2> 2340471 INFO  (jetty-launcher-4154-thread-2) [n:127.0.0.1:36783_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36783.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@639a08ba
   [junit4]   2> 2340471 INFO  (jetty-launcher-4154-thread-2) [n:127.0.0.1:36783_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36783.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@639a08ba
   [junit4]   2> 2340472 INFO  (jetty-launcher-4154-thread-2) [n:127.0.0.1:36783_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest_B018BFF602F6BE9C-001/tempDir-001/node1/.
   [junit4]   2> 2340477 INFO  (jetty-launcher-4154-thread-1) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2340479 INFO  (jetty-launcher-4154-thread-1) [n:127.0.0.1:42028_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2340481 INFO  (jetty-launcher-4154-thread-1) [n:127.0.0.1:42028_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42028_solr
   [junit4]   2> 2340483 INFO  (zkCallback-4170-thread-1-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2340483 INFO  (zkCallback-4172-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2340552 INFO  (jetty-launcher-4154-thread-1) [n:127.0.0.1:42028_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42028.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@639a08ba
   [junit4]   2> 2340558 INFO  (jetty-launcher-4154-thread-1) [n:127.0.0.1:42028_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42028.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@639a08ba
   [junit4]   2> 2340558 INFO  (jetty-launcher-4154-thread-1) [n:127.0.0.1:42028_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42028.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@639a08ba
   [junit4]   2> 2340559 INFO  (jetty-launcher-4154-thread-1) [n:127.0.0.1:42028_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest_B018BFF602F6BE9C-001/tempDir-001/node2/.
   [junit4]   2> 2340594 INFO  (zkConnectionManagerCallback-4179-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2340597 INFO  (zkConnectionManagerCallback-4183-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2340599 INFO  (SUITE-DeleteShardTest-seed#[B018BFF602F6BE9C]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2340600 INFO  (SUITE-DeleteShardTest-seed#[B018BFF602F6BE9C]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46016/solr ready
   [junit4]   2> 2340617 INFO  (TEST-DeleteShardTest.testDirectoryCleanupAfterDeleteShard-seed#[B018BFF602F6BE9C]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDirectoryCleanupAfterDeleteShard
   [junit4]   2> 2340618 INFO  (qtp115708258-24008) [n:127.0.0.1:36783_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params shards=a,b,c&replicationFactor=1&collection.configName=conf&maxShardsPerNode=2&name=deleteshard_test&router.name=implicit&nrtReplicas=1&action=CREATE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2340621 INFO  (OverseerThreadFactory-8003-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.CreateCollectionCmd Create collection deleteshard_test
   [junit4]   2> 2340739 INFO  (OverseerStateUpdate-99302718037819398-127.0.0.1:36783_solr-n_0000000000) [n:127.0.0.1:36783_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"deleteshard_test",
   [junit4]   2>   "shard":"a",
   [junit4]   2>   "core":"deleteshard_test_a_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42028/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 2340743 INFO  (OverseerStateUpdate-99302718037819398-127.0.0.1:36783_solr-n_0000000000) [n:127.0.0.1:36783_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"deleteshard_test",
   [junit4]   2>   "shard":"b",
   [junit4]   2>   "core":"deleteshard_test_b_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:36783/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 2340747 INFO  (OverseerStateUpdate-99302718037819398-127.0.0.1:36783_solr-n_0000000000) [n:127.0.0.1:36783_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"deleteshard_test",
   [junit4]   2>   "shard":"c",
   [junit4]   2>   "core":"deleteshard_test_c_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42028/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 2340950 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&newCollection=true&name=deleteshard_test_c_replica_n4&action=CREATE&numShards=3&collection=deleteshard_test&shard=c&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2340951 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2340952 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=deleteshard_test_a_replica_n1&action=CREATE&numShards=3&collection=deleteshard_test&shard=a&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2340953 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf&newCollection=true&name=deleteshard_test_b_replica_n2&action=CREATE&numShards=3&collection=deleteshard_test&shard=b&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2340953 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2341059 INFO  (zkCallback-4170-thread-1-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2341059 INFO  (zkCallback-4172-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2341059 INFO  (zkCallback-4172-thread-2-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2341060 INFO  (zkCallback-4170-thread-2-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2341967 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2341968 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2341968 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2341974 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.s.IndexSchema [deleteshard_test_a_replica_n1] Schema name=minimal
   [junit4]   2> 2341976 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.s.IndexSchema [deleteshard_test_b_replica_n2] Schema name=minimal
   [junit4]   2> 2341977 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.s.IndexSchema [deleteshard_test_c_replica_n4] Schema name=minimal
   [junit4]   2> 2341977 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2341977 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'deleteshard_test_a_replica_n1' using configuration from collection deleteshard_test, trusted=true
   [junit4]   2> 2341977 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42028.solr.core.deleteshard_test.a.replica_n1' (registry 'solr.core.deleteshard_test.a.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@639a08ba
   [junit4]   2> 2341977 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2341978 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.c.SolrCore [[deleteshard_test_a_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest_B018BFF602F6BE9C-001/tempDir-001/node2/deleteshard_test_a_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest_B018BFF602F6BE9C-001/tempDir-001/node2/./deleteshard_test_a_replica_n1/data/]
   [junit4]   2> 2341978 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2341978 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'deleteshard_test_b_replica_n2' using configuration from collection deleteshard_test, trusted=true
   [junit4]   2> 2341978 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36783.solr.core.deleteshard_test.b.replica_n2' (registry 'solr.core.deleteshard_test.b.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@639a08ba
   [junit4]   2> 2341978 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2341978 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.c.SolrCore [[deleteshard_test_b_replica_n2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest_B018BFF602F6BE9C-001/tempDir-001/node1/deleteshard_test_b_replica_n2], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest_B018BFF602F6BE9C-001/tempDir-001/node1/./deleteshard_test_b_replica_n2/data/]
   [junit4]   2> 2341978 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2341978 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'deleteshard_test_c_replica_n4' using configuration from collection deleteshard_test, trusted=true
   [junit4]   2> 2341979 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42028.solr.core.deleteshard_test.c.replica_n4' (registry 'solr.core.deleteshard_test.c.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@639a08ba
   [junit4]   2> 2341979 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2341979 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.c.SolrCore [[deleteshard_test_c_replica_n4] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest_B018BFF602F6BE9C-001/tempDir-001/node2/deleteshard_test_c_replica_n4], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest_B018BFF602F6BE9C-001/tempDir-001/node2/./deleteshard_test_c_replica_n4/data/]
   [junit4]   2> 2342049 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2342049 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2342050 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2342050 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2342050 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2342050 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2342051 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2342051 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2342052 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@11c40945[deleteshard_test_c_replica_n4] main]
   [junit4]   2> 2342052 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4319c766[deleteshard_test_a_replica_n1] main]
   [junit4]   2> 2342053 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2342053 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2342053 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2342054 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2342054 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2342055 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2342055 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2342055 INFO  (searcherExecutor-8008-thread-1-processing-n:127.0.0.1:42028_solr x:deleteshard_test_a_replica_n1 s:a c:deleteshard_test r:core_node3) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.c.SolrCore [deleteshard_test_a_replica_n1] Registered new searcher Searcher@4319c766[deleteshard_test_a_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2342055 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588843490566995968
   [junit4]   2> 2342056 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@609d0892[deleteshard_test_b_replica_n2] main]
   [junit4]   2> 2342057 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2342057 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2342058 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2342058 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2342059 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2342059 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2342059 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588843490571190272
   [junit4]   2> 2342060 INFO  (searcherExecutor-8010-thread-1-processing-n:127.0.0.1:42028_solr x:deleteshard_test_c_replica_n4 s:c c:deleteshard_test r:core_node6) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.c.SolrCore [deleteshard_test_c_replica_n4] Registered new searcher Searcher@11c40945[deleteshard_test_c_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2342060 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588843490572238848
   [junit4]   2> 2342063 INFO  (searcherExecutor-8009-thread-1-processing-n:127.0.0.1:36783_solr x:deleteshard_test_b_replica_n2 s:b c:deleteshard_test r:core_node5) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.c.SolrCore [deleteshard_test_b_replica_n2] Registered new searcher Searcher@609d0892[deleteshard_test_b_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2342064 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2342064 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2342064 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42028/solr/deleteshard_test_a_replica_n1/
   [junit4]   2> 2342064 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2342064 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:42028/solr/deleteshard_test_a_replica_n1/ has no replicas
   [junit4]   2> 2342064 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2342065 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2342065 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2342065 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36783/solr/deleteshard_test_b_replica_n2/
   [junit4]   2> 2342065 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2342065 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:36783/solr/deleteshard_test_b_replica_n2/ has no replicas
   [junit4]   2> 2342065 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2342067 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2342067 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2342068 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42028/solr/deleteshard_test_c_replica_n4/
   [junit4]   2> 2342068 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2342069 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.c.SyncStrategy http://127.0.0.1:42028/solr/deleteshard_test_c_replica_n4/ has no replicas
   [junit4]   2> 2342069 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2342069 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42028/solr/deleteshard_test_a_replica_n1/ a
   [junit4]   2> 2342072 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36783/solr/deleteshard_test_b_replica_n2/ b
   [junit4]   2> 2342075 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42028/solr/deleteshard_test_c_replica_n4/ c
   [junit4]   2> 2342177 INFO  (zkCallback-4172-thread-2-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2342178 INFO  (zkCallback-4172-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2342178 INFO  (zkCallback-4170-thread-2-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2342178 INFO  (zkCallback-4170-thread-1-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2342221 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2342223 INFO  (qtp1050892156-23998) [n:127.0.0.1:42028_solr c:deleteshard_test s:a r:core_node3 x:deleteshard_test_a_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=deleteshard_test_a_replica_n1&action=CREATE&numShards=3&collection=deleteshard_test&shard=a&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1271
   [junit4]   2> 2342223 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2342225 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteshard_test s:b r:core_node5 x:deleteshard_test_b_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf&newCollection=true&name=deleteshard_test_b_replica_n2&action=CREATE&numShards=3&collection=deleteshard_test&shard=b&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1272
   [junit4]   2> 2342227 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2342230 INFO  (qtp1050892156-23996) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&newCollection=true&name=deleteshard_test_c_replica_n4&action=CREATE&numShards=3&collection=deleteshard_test&shard=c&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1279
   [junit4]   2> 2342233 INFO  (qtp115708258-24008) [n:127.0.0.1:36783_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 2342330 INFO  (zkCallback-4172-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2342331 INFO  (zkCallback-4170-thread-1-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2342331 INFO  (zkCallback-4172-thread-3-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2342331 INFO  (zkCallback-4170-thread-2-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2342624 INFO  (OverseerCollectionConfigSetProcessor-99302718037819398-127.0.0.1:36783_solr-n_0000000000) [n:127.0.0.1:36783_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2343234 INFO  (qtp115708258-24008) [n:127.0.0.1:36783_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={shards=a,b,c&replicationFactor=1&collection.configName=conf&maxShardsPerNode=2&name=deleteshard_test&router.name=implicit&nrtReplicas=1&action=CREATE&wt=javabin&version=2} status=0 QTime=2615
   [junit4]   2> 2343236 INFO  (qtp1050892156-24000) [n:127.0.0.1:42028_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={action=STATUS&indexInfo=true&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2343239 INFO  (qtp115708258-24011) [n:127.0.0.1:36783_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deleteshard with params action=DELETESHARD&collection=deleteshard_test&shard=a&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2343242 INFO  (OverseerThreadFactory-8003-thread-2-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.DeleteShardCmd Delete shard invoked
   [junit4]   2> 2343242 INFO  (OverseerThreadFactory-8003-thread-2-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.DeleteShardCmd Deleting replica for collection=deleteshard_test shard=a on node=127.0.0.1:42028_solr
   [junit4]   2> 2343243 INFO  (qtp1050892156-24001) [n:127.0.0.1:42028_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.deleteshard_test.a.replica_n1, tag=null
   [junit4]   2> 2343243 INFO  (qtp1050892156-24001) [n:127.0.0.1:42028_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2dbab747: rootName = solr_42028, domain = solr.core.deleteshard_test.a.replica_n1, service url = null, agent id = null] for registry solr.core.deleteshard_test.a.replica_n1 / com.codahale.metrics.MetricRegistry@55accc61
   [junit4]   2> 2343252 INFO  (qtp1050892156-24001) [n:127.0.0.1:42028_solr    ] o.a.s.c.SolrCore [deleteshard_test_a_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4a886316
   [junit4]   2> 2343252 INFO  (qtp1050892156-24001) [n:127.0.0.1:42028_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.deleteshard_test.a.replica_n1, tag=1250452246
   [junit4]   2> 2343252 INFO  (qtp1050892156-24001) [n:127.0.0.1:42028_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.deleteshard_test.a.leader, tag=1250452246
   [junit4]   2> 2343255 INFO  (qtp1050892156-24001) [n:127.0.0.1:42028_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=deleteshard_test_a_replica_n1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2&deleteIndex=true} status=0 QTime=12
   [junit4]   2> 2343357 INFO  (zkCallback-4170-thread-1-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2343357 INFO  (zkCallback-4172-thread-3-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2343357 INFO  (zkCallback-4170-thread-2-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2343357 INFO  (zkCallback-4172-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2343459 INFO  (OverseerStateUpdate-99302718037819398-127.0.0.1:36783_solr-n_0000000000) [n:127.0.0.1:36783_solr    ] o.a.s.c.o.CollectionMutator Removing collection: deleteshard_test shard: a from clusterstate
   [junit4]   2> 2343560 INFO  (zkCallback-4172-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2343561 INFO  (zkCallback-4170-thread-1-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2343562 INFO  (zkCallback-4170-thread-2-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2343562 INFO  (zkCallback-4172-thread-3-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2343759 INFO  (OverseerThreadFactory-8003-thread-2-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.DeleteShardCmd Successfully deleted collection: deleteshard_test, shard: a
   [junit4]   2> 2343761 INFO  (qtp115708258-24011) [n:127.0.0.1:36783_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=DELETESHARD&collection=deleteshard_test&shard=a&wt=javabin&version=2} status=0 QTime=522
   [junit4]   2> 2343763 INFO  (qtp115708258-24007) [n:127.0.0.1:36783_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={action=STATUS&indexInfo=true&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2343763 INFO  (qtp115708258-24005) [n:127.0.0.1:36783_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deleteshard with params deleteInstanceDir=false&deleteDataDir=false&action=DELETESHARD&collection=deleteshard_test&shard=b&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2343767 INFO  (OverseerThreadFactory-8003-thread-3-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.DeleteShardCmd Delete shard invoked
   [junit4]   2> 2343767 INFO  (OverseerThreadFactory-8003-thread-3-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.DeleteShardCmd Deleting replica for collection=deleteshard_test shard=b on node=127.0.0.1:36783_solr
   [junit4]   2> 2343768 INFO  (OverseerCollectionConfigSetProcessor-99302718037819398-127.0.0.1:36783_solr-n_0000000000) [n:127.0.0.1:36783_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2343768 INFO  (qtp115708258-24009) [n:127.0.0.1:36783_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.deleteshard_test.b.replica_n2, tag=null
   [junit4]   2> 2343768 INFO  (qtp115708258-24009) [n:127.0.0.1:36783_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5f1a72fa: rootName = solr_36783, domain = solr.core.deleteshard_test.b.replica_n2, service url = null, agent id = null] for registry solr.core.deleteshard_test.b.replica_n2 / com.codahale.metrics.MetricRegistry@48b704cd
   [junit4]   2> 2343776 INFO  (qtp115708258-24009) [n:127.0.0.1:36783_solr    ] o.a.s.c.SolrCore [deleteshard_test_b_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@b4437d
   [junit4]   2> 2343776 INFO  (qtp115708258-24009) [n:127.0.0.1:36783_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.deleteshard_test.b.replica_n2, tag=11813757
   [junit4]   2> 2343776 INFO  (qtp115708258-24009) [n:127.0.0.1:36783_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.deleteshard_test.b.leader, tag=11813757
   [junit4]   2> 2343778 INFO  (qtp115708258-24009) [n:127.0.0.1:36783_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=false&core=deleteshard_test_b_replica_n2&qt=/admin/cores&deleteDataDir=false&action=UNLOAD&wt=javabin&version=2&deleteIndex=true} status=0 QTime=10
   [junit4]   2> 2343880 INFO  (zkCallback-4170-thread-1-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2343880 INFO  (zkCallback-4170-thread-2-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2343983 INFO  (OverseerStateUpdate-99302718037819398-127.0.0.1:36783_solr-n_0000000000) [n:127.0.0.1:36783_solr    ] o.a.s.c.o.CollectionMutator Removing collection: deleteshard_test shard: b from clusterstate
   [junit4]   2> 2344084 INFO  (zkCallback-4170-thread-1-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2344084 INFO  (zkCallback-4170-thread-2-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteshard_test/state.json] for collection [deleteshard_test] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2344292 INFO  (OverseerThreadFactory-8003-thread-3-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.DeleteShardCmd Successfully deleted collection: deleteshard_test, shard: b
   [junit4]   2> 2344298 INFO  (qtp115708258-24005) [n:127.0.0.1:36783_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={deleteInstanceDir=false&deleteDataDir=false&action=DELETESHARD&collection=deleteshard_test&shard=b&wt=javabin&version=2} status=0 QTime=534
   [junit4]   2> 2344306 INFO  (TEST-DeleteShardTest.testDirectoryCleanupAfterDeleteShard-seed#[B018BFF602F6BE9C]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDirectoryCleanupAfterDeleteShard
   [junit4]   2> 2344326 INFO  (TEST-DeleteShardTest.test-seed#[B018BFF602F6BE9C]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 2344327 INFO  (qtp115708258-24003) [n:127.0.0.1:36783_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=deleteShard&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2344329 INFO  (OverseerThreadFactory-8003-thread-4-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.CreateCollectionCmd Create collection deleteShard
   [junit4]   2> 2344330 INFO  (OverseerCollectionConfigSetProcessor-99302718037819398-127.0.0.1:36783_solr-n_0000000000) [n:127.0.0.1:36783_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2344540 INFO  (OverseerStateUpdate-99302718037819398-127.0.0.1:36783_solr-n_0000000000) [n:127.0.0.1:36783_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"deleteShard",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"deleteShard_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:36783/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 2344546 INFO  (OverseerStateUpdate-99302718037819398-127.0.0.1:36783_solr-n_0000000000) [n:127.0.0.1:36783_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"deleteShard",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"deleteShard_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42028/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 2344747 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=deleteShard_shard2_replica_n2&action=CREATE&numShards=2&collection=deleteShard&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2344748 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=deleteShard_shard1_replica_n1&action=CREATE&numShards=2&collection=deleteShard&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2344855 INFO  (zkCallback-4172-thread-3-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteShard/state.json] for collection [deleteShard] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2344855 INFO  (zkCallback-4170-thread-2-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteShard/state.json] for collection [deleteShard] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2344855 INFO  (zkCallback-4172-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteShard/state.json] for collection [deleteShard] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2344855 INFO  (zkCallback-4170-thread-1-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteShard/state.json] for collection [deleteShard] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2345630 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2345630 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2345635 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.s.IndexSchema [deleteShard_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 2345636 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.s.IndexSchema [deleteShard_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 2345636 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2345636 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'deleteShard_shard1_replica_n1' using configuration from collection deleteShard, trusted=true
   [junit4]   2> 2345637 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36783.solr.core.deleteShard.shard1.replica_n1' (registry 'solr.core.deleteShard.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@639a08ba
   [junit4]   2> 2345637 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2345637 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.c.SolrCore [[deleteShard_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest_B018BFF602F6BE9C-001/tempDir-001/node1/deleteShard_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest_B018BFF602F6BE9C-001/tempDir-001/node1/./deleteShard_shard1_replica_n1/data/]
   [junit4]   2> 2345638 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2345638 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'deleteShard_shard2_replica_n2' using configuration from collection deleteShard, trusted=true
   [junit4]   2> 2345638 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42028.solr.core.deleteShard.shard2.replica_n2' (registry 'solr.core.deleteShard.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@639a08ba
   [junit4]   2> 2345638 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2345638 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.c.SolrCore [[deleteShard_shard2_replica_n2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest_B018BFF602F6BE9C-001/tempDir-001/node2/deleteShard_shard2_replica_n2], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest_B018BFF602F6BE9C-001/tempDir-001/node2/./deleteShard_shard2_replica_n2/data/]
   [junit4]   2> 2345683 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2345683 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2345684 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2345684 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2345685 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@1df0a647[deleteShard_shard2_replica_n2] main]
   [junit4]   2> 2345686 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2345686 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2345687 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2345687 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588843494375424000
   [junit4]   2> 2345692 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2345692 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2345692 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42028/solr/deleteShard_shard2_replica_n2/
   [junit4]   2> 2345692 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2345692 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:42028/solr/deleteShard_shard2_replica_n2/ has no replicas
   [junit4]   2> 2345692 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2345696 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42028/solr/deleteShard_shard2_replica_n2/ shard2
   [junit4]   2> 2345697 INFO  (searcherExecutor-8024-thread-1-processing-n:127.0.0.1:42028_solr x:deleteShard_shard2_replica_n2 s:shard2 c:deleteShard r:core_node4) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.c.SolrCore [deleteShard_shard2_replica_n2] Registered new searcher Searcher@1df0a647[deleteShard_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2345699 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2345699 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2345700 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2345700 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2345701 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@36db3213[deleteShard_shard1_replica_n1] main]
   [junit4]   2> 2345703 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2345703 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2345704 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2345704 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588843494393249792
   [junit4]   2> 2345704 INFO  (searcherExecutor-8023-thread-1-processing-n:127.0.0.1:36783_solr x:deleteShard_shard1_replica_n1 s:shard1 c:deleteShard r:core_node3) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.c.SolrCore [deleteShard_shard1_replica_n1] Registered new searcher Searcher@36db3213[deleteShard_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2345709 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2345709 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2345709 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36783/solr/deleteShard_shard1_replica_n1/
   [junit4]   2> 2345709 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2345709 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:36783/solr/deleteShard_shard1_replica_n1/ has no replicas
   [junit4]   2> 2345709 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2345712 INFO  (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36783/solr/deleteShard_shard1_replica_n1/ shard1
   [junit4]   2> 2345814 INFO  (zkCallback-4172-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteShard/state.json] for collection [deleteShard] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2345814 INFO  (zkCallback-4170-thread-2-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteShard/state.json] for collection [deleteShard] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2345814 INFO  (zkCallback-4170-thread-1-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteShard/state.json] for collection [deleteShard] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2345814 INFO  (zkCallback-4172-thread-3-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deleteShard/state.json] for collection [deleteShard] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2345848 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2345850 INFO  (qtp1050892156-24002) [n:127.0.0.1:42028_solr c:deleteShard s:shard2 r:core_node4 x:deleteShard_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=deleteShard_shard2_replica_n2&action=CREATE&numShards=2&collection=deleteShard&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1233
   [junit4]   2> 2345851 ERROR (OverseerThreadFactory-8003-thread-4-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.OverseerCollectionMessageHandler Error from shard: http://127.0.0.1:36783/solr
   [junit4]   2> org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:36783/solr
   [junit4]   2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
   [junit4]   2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
   [junit4]   2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:172)
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]   2> at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]   2> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
   [junit4]   2> at java.net.SocketInputStream.read(SocketInputStream.java:171)
   [junit4]   2> at java.net.SocketInputStream.read(SocketInputStream.java:141)
   [junit4]   2> at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
   [junit4]   2> at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
   [junit4]   2> at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
   [junit4]   2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
   [junit4]   2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
   [junit4]   2> at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
   [junit4]   2> at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
   [junit4]   2> at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
   [junit4]   2> at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
   [junit4]   2> at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
   [junit4]   2> at org.apache.solr.util.stats.InstrumentedHttpRequestExecutor.execute(InstrumentedHttpRequestExecutor.java:118)
   [junit4]   2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
   [junit4]   2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
   [junit4]   2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
   [junit4]   2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
   [junit4]   2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
   [junit4]   2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
   [junit4]   2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
   [junit4]   2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
   [junit4]   2> ... 12 more
   [junit4]   2> 2345852 ERROR (OverseerThreadFactory-8003-thread-4-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.OverseerCollectionMessageHandler Cleaning up collection [deleteShard].
   [junit4]   2> 2345853 INFO  (OverseerThreadFactory-8003-thread-4-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 2345854 INFO  (qtp1050892156-24004) [n:127.0.0.1:42028_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.deleteShard.shard2.replica_n2, tag=null
   [junit4]   2> 2345854 INFO  (qtp115708258-24008) [n:127.0.0.1:36783_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.deleteShard.shard1.replica_n1, tag=null
   [junit4]   2> 2345854 INFO  (qtp115708258-24008) [n:127.0.0.1:36783_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@391b5051: rootName = solr_36783, domain = solr.core.deleteShard.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.deleteShard.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@69403bf6
   [junit4]   2> 2345854 INFO  (qtp1050892156-24004) [n:127.0.0.1:42028_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2bb29dbe: rootName = solr_42028, domain = solr.core.deleteShard.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.deleteShard.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@30af66b7
   [junit4]   2> 2345869 INFO  (qtp115708258-24008) [n:127.0.0.1:36783_solr    ] o.a.s.c.SolrCore [deleteShard_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@7be48cea
   [junit4]   2> 2345869 INFO  (qtp115708258-24008) [n:127.0.0.1:36783_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.deleteShard.shard1.replica_n1, tag=2078575850
   [junit4]   2> 2345869 INFO  (qtp115708258-24008) [n:127.0.0.1:36783_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.deleteShard.shard1.leader, tag=2078575850
   [junit4]   2> 2345872 INFO  (qtp115708258-24008) [n:127.0.0.1:36783_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=deleteShard_shard1_replica_n1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=18
   [junit4]   2> 2345875 INFO  (qtp1050892156-24004) [n:127.0.0.1:42028_solr    ] o.a.s.c.SolrCore [deleteShard_shard2_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@88d30f8
   [junit4]   2> 2345875 INFO  (qtp1050892156-24004) [n:127.0.0.1:42028_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.deleteShard.shard2.replica_n2, tag=143470840
   [junit4]   2> 2345875 INFO  (qtp1050892156-24004) [n:127.0.0.1:42028_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.deleteShard.shard2.leader, tag=143470840
   [junit4]   2> 2345878 INFO  (qtp1050892156-24004) [n:127.0.0.1:42028_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=deleteShard_shard2_replica_n2&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=25
   [junit4]   2> 2345881 ERROR (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.c.ZkContainer
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2> at org.apache.solr.cloud.ZkController.publish(ZkController.java:1350)
   [junit4]   2> at org.apache.solr.cloud.ZkController.publish(ZkController.java:1342)
   [junit4]   2> at org.apache.solr.cloud.ZkController.publish(ZkController.java:1338)
   [junit4]   2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:188)
   [junit4]   2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:208)
   [junit4]   2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:892)
   [junit4]   2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1053)
   [junit4]   2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:954)
   [junit4]   2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
   [junit4]   2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
   [junit4]   2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
   [junit4]   2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
   [junit4]   2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
   [junit4]   2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
   [junit4]   2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 2345882 ERROR (qtp115708258-24010) [n:127.0.0.1:36783_solr c:deleteShard s:shard1 r:core_node3 x:deleteShard_shard1_replica_n1] o.a.s.c.ZkContainer :java.lang.NullPointerException
   [junit4]   2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1095)
   [junit4]   2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1014)
   [junit4]   2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:181)
   [junit4]   2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:208)
   [junit4]   2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:892)
   [junit4]   2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1053)
   [junit4]   2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:954)
   [junit4]   2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
   [junit4]   2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
   [junit4]   2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
   [junit4]   2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
   [junit4]   2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doF

[...truncated too long message...]

n_0000000000) [n:127.0.0.1:36783_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36783_solr
   [junit4]   2> 2376690 INFO  (zkCallback-4170-thread-2-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 2376691 INFO  (zkCallback-4172-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 2376691 WARN  (OverseerAutoScalingTriggerThread-99302718037819398-127.0.0.1:36783_solr-n_0000000000) [n:127.0.0.1:36783_solr    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 2376693 INFO  (jetty-closer-4155-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 2376694 INFO  (zkCallback-4170-thread-1-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 2376694 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x160cb4bc1e00006, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 2376697 INFO  (coreCloseExecutor-8034-thread-1) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.c.SolrCore [deleteshard_test_c_replica_n4]  CLOSING SolrCore org.apache.solr.core.SolrCore@460270bf
   [junit4]   2> 2376697 INFO  (coreCloseExecutor-8034-thread-1) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.deleteshard_test.c.replica_n4, tag=1174565055
   [junit4]   2> 2376697 INFO  (coreCloseExecutor-8034-thread-1) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2f65e95c: rootName = solr_42028, domain = solr.core.deleteshard_test.c.replica_n4, service url = null, agent id = null] for registry solr.core.deleteshard_test.c.replica_n4 / com.codahale.metrics.MetricRegistry@3cc2c7fb
   [junit4]   2> 2376697 INFO  (jetty-closer-4155-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@72e834c9{/solr,null,UNAVAILABLE}
   [junit4]   2> 2376698 INFO  (zkCallback-4170-thread-2-processing-n:127.0.0.1:42028_solr) [n:127.0.0.1:42028_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42028_solr
   [junit4]   2> 2376706 INFO  (coreCloseExecutor-8034-thread-1) [n:127.0.0.1:42028_solr c:deleteshard_test s:c r:core_node6 x:deleteshard_test_c_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.deleteshard_test.c.leader, tag=1174565055
   [junit4]   2> 2376711 INFO  (jetty-closer-4155-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@333c4225{/solr,null,UNAVAILABLE}
   [junit4]   2> 2376713 ERROR (SUITE-DeleteShardTest-seed#[B018BFF602F6BE9C]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2376714 INFO  (SUITE-DeleteShardTest-seed#[B018BFF602F6BE9C]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46016 46016
   [junit4]   2> 2383274 INFO  (Thread-8824) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46016 46016
   [junit4]   2> 2383275 WARN  (Thread-8824) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/aliases.json
   [junit4]   2> 3 /solr/configs/conf
   [junit4]   2> 3 /solr/clusterprops.json
   [junit4]   2> 2 /solr/security.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 15 /solr/collections/deleteshard_test/state.json
   [junit4]   2> 8 /solr/collections/deleteShard/state.json
   [junit4]   2> 3 /solr/clusterstate.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/live_nodes
   [junit4]   2> 3 /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest_B018BFF602F6BE9C-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=249, maxMBSortInHeap=5.073923016233536, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@7b253c93), locale=de, timezone=US/Samoa
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=153600480,total=515899392
   [junit4]   2> NOTE: All tests run in this JVM: [CoreSorterTest, TestAddFieldRealTimeGet, TestShardHandlerFactory, DataDrivenBlockJoinTest, TestRestoreCore, SyncSliceTest, RuleEngineTest, TestRawTransformer, TestSubQueryTransformer, TestConfigSetProperties, TestUtilizeNode, TestSkipOverseerOperations, DirectUpdateHandlerOptimizeTest, PropertiesRequestHandlerTest, TestSolrDeletionPolicy2, TestJsonRequest, TestCorePropertiesReload, TestUnifiedSolrHighlighter, TestManagedResourceStorage, OverseerRolesTest, HdfsTlogReplayBufferedWhileIndexingTest, LeaderInitiatedRecoveryOnCommitTest, TestSubQueryTransformerDistrib, DirectSolrConnectionTest, RequiredFieldsTest, SolrRequestParserTest, TestIntervalFaceting, TestEmbeddedSolrServerSchemaAPI, UUIDFieldTest, TriLevelCompositeIdRoutingTest, BitVectorTest, TestFieldCacheSortRandom, TestFuzzyAnalyzedSuggestions, TestAuthenticationFramework, TestSolrQueryResponse, TestSmileRequest, TestSolrCloudWithKerberosAlt, XmlUpdateRequestHandlerTest, DirectUpdateHandlerTest, TestCodecSupport, ZkFailoverTest, HdfsDirectoryTest, TestSurroundQueryParser, MissingSegmentRecoveryTest, TestPhraseSuggestions, SpatialHeatmapFacetsTest, DistributedQueryComponentOptimizationTest, TestQuerySenderListener, TestLeaderInitiatedRecoveryThread, TestInPlaceUpdatesDistrib, TestDistribDocBasedVersion, BaseCdcrDistributedZkTest, BinaryUpdateRequestHandlerTest, StatsComponentTest, TestFieldCollectionResource, TestDynamicFieldCollectionResource, TestSolrCoreProperties, TestConfigSetsAPIExclusivity, SolrTestCaseJ4Test, ShardSplitTest, SuggesterWFSTTest, TestCursorMarkWithoutUniqueKey, IgnoreCommitOptimizeUpdateProcessorFactoryTest, HdfsNNFailoverTest, TestNodeLostTrigger, TestBinaryField, SolrCoreMetricManagerTest, TestCryptoKeys, TestHighlightDedupGrouping, TestCloudPivotFacet, MoveReplicaHDFSTest, SolrMetricReporterTest, ConvertedLegacyTest, ExitableDirectoryReaderTest, TestUseDocValuesAsStored2, RequestLoggingTest, FieldAnalysisRequestHandlerTest, SolrJmxReporterCloudTest, TestSSLRandomization, HdfsAutoAddReplicasIntegrationTest, BlockDirectoryTest, TestDFISimilarityFactory, TestSolrConfigHandler, TestSolrCloudSnapshots, TestFiltering, SuggesterFSTTest, NoCacheHeaderTest, TestJsonFacetRefinement, SuggesterTest, NumericFieldsTest, TestCloudNestedDocsSort, UUIDUpdateProcessorFallbackTest, TestReplicationHandler, TestPHPSerializedResponseWriter, ClusterStateUpdateTest, FacetPivotSmallTest, AddSchemaFieldsUpdateProcessorFactoryTest, LeaderElectionTest, TestAtomicUpdateErrorCases, TestTrieFacet, LegacyCloudClusterPropTest, TestClusterStateMutator, TestJmxIntegration, TestSolrCoreParser, UninvertDocValuesMergePolicyTest, MinimalSchemaTest, CoreAdminRequestStatusTest, TestApiFramework, DistribJoinFromCollectionTest, ShardRoutingTest, UtilsToolTest, TestFieldTypeCollectionResource, TestXIncludeConfig, PrimUtilsTest, TestInfoStreamLogging, TestPointFields, HdfsRecoveryZkTest, TestDFRSimilarityFactory, TestQueryWrapperFilter, TestCSVResponseWriter, AutoscalingHistoryHandlerTest, TestLeaderElectionWithEmptyReplica, HdfsSyncSliceTest, AsyncCallRequestStatusResponseTest, DistributedFacetPivotWhiteBoxTest, AtomicUpdatesTest, MetricUtilsTest, CircularListTest, TestJoin, RecoveryAfterSoftCommitTest, TestRandomFlRTGCloud, TestEmbeddedSolrServerAdminHandler, ReplaceNodeTest, TestBinaryResponseWriter, HighlighterConfigTest, TestReplicationHandlerBackup, TestQuerySenderNoQuery, OverseerTest, SSLMigrationTest, SliceStateTest, HLLUtilTest, TestNoOpRegenerator, SortByFunctionTest, ForceLeaderTest, AnalysisErrorHandlingTest, SubstringBytesRefFilterTest, TestShortCircuitedRequests, BasicDistributedZkTest, BadIndexSchemaTest, TestHashQParserPlugin, TestAuthorizationFramework, TestUpdate, CachingDirectoryFactoryTest, TestScoreJoinQPNoScore, HdfsWriteToMultipleCollectionsTest, ScriptEngineTest, TestStressCloudBlindAtomicUpdates, TestBulkSchemaConcurrent, DefaultValueUpdateProcessorTest, TestCloudRecovery, TestUninvertingReader, TestClusterProperties, TestLegacyFieldReuse, DistributedQueryElevationComponentTest, SegmentsInfoRequestHandlerTest, InfoHandlerTest, TestIndexingPerformance, TestSimDistributedQueue, TestStandardQParsers, TestRequestForwarding, CdcrReplicationHandlerTest, SolrInfoBeanTest, TestSchemaNameResource, ShowFileRequestHandlerTest, RegexBoostProcessorTest, TestComplexPhraseLeadingWildcard, CheckHdfsIndexTest, SuggesterTSTTest, WordBreakSolrSpellCheckerTest, FullHLLTest, ResourceLoaderTest, ManagedSchemaRoundRobinCloudTest, UnloadDistributedZkTest, TestSearcherReuse, DeleteShardTest]
   [junit4] Completed [408/766 (1!)] on J0 in 43.13s, 2 tests, 1 error <<< FAILURES!

[...truncated 49257 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1610 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1610/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.core.TestDynamicLoading.testDynamicLoading

Error Message:


Stack Trace:
java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([F7D6A333BE924370:2F9B8E64494FE6D0]:0)
        at org.junit.Assert.fail(Assert.java:92)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNotNull(Assert.java:526)
        at org.junit.Assert.assertNotNull(Assert.java:537)
        at org.apache.solr.core.TestDynamicLoading.testDynamicLoading(TestDynamicLoading.java:97)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13699 lines...]
   [junit4] Suite: org.apache.solr.core.TestDynamicLoading
   [junit4]   2> 4758244 INFO  (SUITE-TestDynamicLoading-seed#[F7D6A333BE924370]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/init-core-data-001
   [junit4]   2> 4758245 INFO  (SUITE-TestDynamicLoading-seed#[F7D6A333BE924370]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 4758246 INFO  (SUITE-TestDynamicLoading-seed#[F7D6A333BE924370]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 4758246 INFO  (SUITE-TestDynamicLoading-seed#[F7D6A333BE924370]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 4758260 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 4758261 INFO  (Thread-8482) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 4758261 INFO  (Thread-8482) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 4758265 ERROR (Thread-8482) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 4758362 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.ZkTestServer start zk server on port:51014
   [junit4]   2> 4758369 INFO  (zkConnectionManagerCallback-7135-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4758380 INFO  (zkConnectionManagerCallback-7137-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4758387 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 4758389 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 4758391 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4758393 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 4758394 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 4758396 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 4758398 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 4758399 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4758401 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4758402 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 4758404 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 4758406 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 4758559 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 4758560 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3ed977a6{/,null,AVAILABLE}
   [junit4]   2> 4758561 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5bb4f8f1{HTTP/1.1,[http/1.1]}{127.0.0.1:33744}
   [junit4]   2> 4758561 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.Server Started @4763172ms
   [junit4]   2> 4758561 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/tempDir-001/control/data, hostContext=/, hostPort=33744, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/control-001/cores}
   [junit4]   2> 4758561 ERROR (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 4758561 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 4758561 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 4758561 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 4758562 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-07T00:37:55.422Z
   [junit4]   2> 4758567 INFO  (zkConnectionManagerCallback-7139-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4758568 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4758568 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/control-001/solr.xml
   [junit4]   2> 4758571 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 4758571 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 4758572 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 4758575 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51014/solr
   [junit4]   2> 4758580 INFO  (zkConnectionManagerCallback-7143-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4758589 INFO  (zkConnectionManagerCallback-7145-thread-1-processing-n:127.0.0.1:33744_) [n:127.0.0.1:33744_    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4758703 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:33744_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4758703 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:33744_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33744_
   [junit4]   2> 4758705 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:33744_    ] o.a.s.c.Overseer Overseer (id=99305748897595396-127.0.0.1:33744_-n_0000000000) starting
   [junit4]   2> 4758713 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:33744_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33744_
   [junit4]   2> 4758719 INFO  (OverseerStateUpdate-99305748897595396-127.0.0.1:33744_-n_0000000000) [n:127.0.0.1:33744_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 4758827 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:33744_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1
   [junit4]   2> 4758834 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:33744_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1
   [junit4]   2> 4758834 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:33744_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1
   [junit4]   2> 4758835 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:33744_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/control-001/cores
   [junit4]   2> 4758851 INFO  (zkConnectionManagerCallback-7151-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4758852 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 4758854 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51014/solr ready
   [junit4]   2> 4758856 INFO  (qtp215995676-34326) [n:127.0.0.1:33744_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:33744_&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4758861 INFO  (OverseerThreadFactory-12310-thread-1-processing-n:127.0.0.1:33744_) [n:127.0.0.1:33744_    ] o.a.s.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 4758985 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 4758987 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 4759128 INFO  (zkCallback-7144-thread-1-processing-n:127.0.0.1:33744_) [n:127.0.0.1:33744_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 4760039 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 4760069 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 4760165 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 4760188 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 4760188 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1
   [junit4]   2> 4760188 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 4760189 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 4760190 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=24, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 4760193 WARN  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 4760228 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 4760228 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 4760229 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 4760229 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 4760230 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=38.16796875, floorSegmentMB=1.3203125, forceMergeDeletesPctAllowed=7.580967010430552, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7145361062711567
   [junit4]   2> 4760230 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3e03c80f[control_collection_shard1_replica_n1] main]
   [junit4]   2> 4760231 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 4760232 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4760232 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 4760233 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588891984428269568
   [junit4]   2> 4760239 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 4760239 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 4760239 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33744/control_collection_shard1_replica_n1/
   [junit4]   2> 4760239 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 4760239 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:33744/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 4760239 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 4760247 INFO  (searcherExecutor-12313-thread-1-processing-n:127.0.0.1:33744_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@3e03c80f[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4760248 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33744/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 4760351 INFO  (zkCallback-7144-thread-1-processing-n:127.0.0.1:33744_) [n:127.0.0.1:33744_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 4760401 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 4760410 INFO  (qtp215995676-34328) [n:127.0.0.1:33744_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1425
   [junit4]   2> 4760433 INFO  (qtp215995676-34326) [n:127.0.0.1:33744_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 4760524 INFO  (zkCallback-7144-thread-1-processing-n:127.0.0.1:33744_) [n:127.0.0.1:33744_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 4760873 INFO  (OverseerCollectionConfigSetProcessor-99305748897595396-127.0.0.1:33744_-n_0000000000) [n:127.0.0.1:33744_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 4761434 INFO  (qtp215995676-34326) [n:127.0.0.1:33744_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:33744_&wt=javabin&version=2} status=0 QTime=2579
   [junit4]   2> 4761438 INFO  (zkConnectionManagerCallback-7155-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4761440 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 4761443 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51014/solr ready
   [junit4]   2> 4761443 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 4761444 INFO  (qtp215995676-34331) [n:127.0.0.1:33744_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4761447 INFO  (OverseerThreadFactory-12310-thread-2-processing-n:127.0.0.1:33744_) [n:127.0.0.1:33744_    ] o.a.s.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 4761448 WARN  (OverseerThreadFactory-12310-thread-2-processing-n:127.0.0.1:33744_) [n:127.0.0.1:33744_    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 4761654 INFO  (qtp215995676-34331) [n:127.0.0.1:33744_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 4761654 INFO  (qtp215995676-34331) [n:127.0.0.1:33744_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=210
   [junit4]   2> 4761786 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-1-001 of type NRT
   [junit4]   2> 4761787 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 4761788 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@533d8b79{/,null,AVAILABLE}
   [junit4]   2> 4761788 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@48df3e1{HTTP/1.1,[http/1.1]}{127.0.0.1:63227}
   [junit4]   2> 4761788 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.Server Started @4766401ms
   [junit4]   2> 4761788 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=63227, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-1-001/cores}
   [junit4]   2> 4762021 ERROR (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 4762021 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 4762024 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 4762024 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 4762024 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-07T00:37:58.884Z
   [junit4]   2> 4762029 INFO  (zkConnectionManagerCallback-7157-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4762031 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4762031 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-1-001/solr.xml
   [junit4]   2> 4762035 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 4762035 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 4762036 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 4762039 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51014/solr
   [junit4]   2> 4762041 INFO  (zkConnectionManagerCallback-7161-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4762044 INFO  (zkConnectionManagerCallback-7163-thread-1-processing-n:127.0.0.1:63227_) [n:127.0.0.1:63227_    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4762055 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:63227_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 4762058 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:63227_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4762061 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:63227_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63227_
   [junit4]   2> 4762062 INFO  (zkCallback-7144-thread-1-processing-n:127.0.0.1:33744_) [n:127.0.0.1:33744_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 4762065 INFO  (zkCallback-7154-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 4762065 INFO  (zkCallback-7162-thread-1-processing-n:127.0.0.1:63227_) [n:127.0.0.1:63227_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 4762136 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:63227_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1
   [junit4]   2> 4762143 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:63227_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1
   [junit4]   2> 4762143 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:63227_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1
   [junit4]   2> 4762144 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:63227_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-1-001/cores
   [junit4]   2> 4762164 INFO  (qtp215995676-34337) [n:127.0.0.1:33744_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:63227_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4762168 INFO  (OverseerCollectionConfigSetProcessor-99305748897595396-127.0.0.1:33744_-n_0000000000) [n:127.0.0.1:33744_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 4762169 INFO  (OverseerThreadFactory-12310-thread-3-processing-n:127.0.0.1:33744_) [n:127.0.0.1:33744_    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:63227_ for creating new replica
   [junit4]   2> 4762173 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 4762173 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 4762281 INFO  (zkCallback-7162-thread-1-processing-n:127.0.0.1:63227_) [n:127.0.0.1:63227_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 4763218 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 4763245 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema [collection1_shard2_replica_n41] Schema name=test
   [junit4]   2> 4763339 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 4763361 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n41' using configuration from collection collection1, trusted=true
   [junit4]   2> 4763361 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n41' (registry 'solr.core.collection1.shard2.replica_n41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1
   [junit4]   2> 4763362 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 4763362 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [[collection1_shard2_replica_n41] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-1-001/cores/collection1_shard2_replica_n41], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-1-001/cores/collection1_shard2_replica_n41/data/]
   [junit4]   2> 4763363 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=24, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 4763366 WARN  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 4763415 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 4763415 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 4763416 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 4763416 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 4763417 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=38.16796875, floorSegmentMB=1.3203125, forceMergeDeletesPctAllowed=7.580967010430552, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7145361062711567
   [junit4]   2> 4763417 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.SolrIndexSearcher Opening [Searcher@66cb2fa5[collection1_shard2_replica_n41] main]
   [junit4]   2> 4763419 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 4763420 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4763420 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 4763420 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588891987770081280
   [junit4]   2> 4763421 INFO  (searcherExecutor-12324-thread-1-processing-n:127.0.0.1:63227_ x:collection1_shard2_replica_n41 s:shard2 c:collection1) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [collection1_shard2_replica_n41] Registered new searcher Searcher@66cb2fa5[collection1_shard2_replica_n41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4763429 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 4763429 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 4763429 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:63227/collection1_shard2_replica_n41/
   [junit4]   2> 4763429 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 4763429 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy http://127.0.0.1:63227/collection1_shard2_replica_n41/ has no replicas
   [junit4]   2> 4763429 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 4763435 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:63227/collection1_shard2_replica_n41/ shard2
   [junit4]   2> 4763543 INFO  (zkCallback-7162-thread-1-processing-n:127.0.0.1:63227_) [n:127.0.0.1:63227_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 4763613 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 4763672 INFO  (qtp1850474082-34377) [n:127.0.0.1:63227_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1499
   [junit4]   2> 4763680 INFO  (qtp215995676-34337) [n:127.0.0.1:33744_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:63227_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1516
   [junit4]   2> 4763749 INFO  (zkCallback-7162-thread-1-processing-n:127.0.0.1:63227_) [n:127.0.0.1:63227_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 4763858 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-2-001 of type NRT
   [junit4]   2> 4763858 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 4763860 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2799c157{/,null,AVAILABLE}
   [junit4]   2> 4763860 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@56bcecf{HTTP/1.1,[http/1.1]}{127.0.0.1:46006}
   [junit4]   2> 4763860 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.Server Started @4768473ms
   [junit4]   2> 4763860 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=46006, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-2-001/cores}
   [junit4]   2> 4763864 ERROR (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 4763864 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 4763864 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 4763865 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 4763865 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-07T00:38:00.725Z
   [junit4]   2> 4763868 INFO  (zkConnectionManagerCallback-7167-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4763870 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4763870 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-2-001/solr.xml
   [junit4]   2> 4763874 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 4763874 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 4763875 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 4763877 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51014/solr
   [junit4]   2> 4763879 INFO  (zkConnectionManagerCallback-7171-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4763884 INFO  (zkConnectionManagerCallback-7173-thread-1-processing-n:127.0.0.1:46006_) [n:127.0.0.1:46006_    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4763892 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:46006_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 4763895 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:46006_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4763902 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:46006_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46006_
   [junit4]   2> 4763902 INFO  (zkCallback-7144-thread-1-processing-n:127.0.0.1:33744_) [n:127.0.0.1:33744_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 4763902 INFO  (zkCallback-7154-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 4763902 INFO  (zkCallback-7162-thread-1-processing-n:127.0.0.1:63227_) [n:127.0.0.1:63227_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 4763908 INFO  (zkCallback-7172-thread-1-processing-n:127.0.0.1:46006_) [n:127.0.0.1:46006_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 4763968 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:46006_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1
   [junit4]   2> 4763976 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:46006_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1
   [junit4]   2> 4763976 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:46006_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1
   [junit4]   2> 4763977 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:46006_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-2-001/cores
   [junit4]   2> 4764006 INFO  (qtp215995676-34336) [n:127.0.0.1:33744_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:46006_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4764008 INFO  (OverseerCollectionConfigSetProcessor-99305748897595396-127.0.0.1:33744_-n_0000000000) [n:127.0.0.1:33744_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 4764009 INFO  (OverseerThreadFactory-12310-thread-4-processing-n:127.0.0.1:33744_) [n:127.0.0.1:33744_    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:46006_ for creating new replica
   [junit4]   2> 4764013 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 4764014 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 4764119 INFO  (zkCallback-7162-thread-1-processing-n:127.0.0.1:63227_) [n:127.0.0.1:63227_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 4764120 INFO  (zkCallback-7172-thread-1-processing-n:127.0.0.1:46006_) [n:127.0.0.1:46006_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 4765050 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 4765082 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema [collection1_shard1_replica_n43] Schema name=test
   [junit4]   2> 4765157 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 4765175 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n43' using configuration from collection collection1, trusted=true
   [junit4]   2> 4765175 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n43' (registry 'solr.core.collection1.shard1.replica_n43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1
   [junit4]   2> 4765176 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 4765176 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [[collection1_shard1_replica_n43] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-2-001/cores/collection1_shard1_replica_n43], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-2-001/cores/collection1_shard1_replica_n43/data/]
   [junit4]   2> 4765177 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=24, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 4765179 WARN  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 4765213 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 4765213 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 4765214 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 4765214 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 4765214 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=38.16796875, floorSegmentMB=1.3203125, forceMergeDeletesPctAllowed=7.580967010430552, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7145361062711567
   [junit4]   2> 4765215 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.SolrIndexSearcher Opening [Searcher@681cb035[collection1_shard1_replica_n43] main]
   [junit4]   2> 4765216 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 4765216 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4765217 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 4765217 INFO  (searcherExecutor-12335-thread-1-processing-n:127.0.0.1:46006_ x:collection1_shard1_replica_n43 s:shard1 c:collection1) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [collection1_shard1_replica_n43] Registered new searcher Searcher@681cb035[collection1_shard1_replica_n43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4765218 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588891989655420928
   [junit4]   2> 4765225 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 4765225 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 4765225 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46006/collection1_shard1_replica_n43/
   [junit4]   2> 4765225 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 4765225 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy http://127.0.0.1:46006/collection1_shard1_replica_n43/ has no replicas
   [junit4]   2> 4765225 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 4765229 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46006/collection1_shard1_replica_n43/ shard1
   [junit4]   2> 4765332 INFO  (zkCallback-7162-thread-1-processing-n:127.0.0.1:63227_) [n:127.0.0.1:63227_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 4765332 INFO  (zkCallback-7172-thread-1-processing-n:127.0.0.1:46006_) [n:127.0.0.1:46006_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 4765382 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 4765398 INFO  (qtp677030743-34408) [n:127.0.0.1:46006_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1385
   [junit4]   2> 4765410 INFO  (qtp215995676-34336) [n:127.0.0.1:33744_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:46006_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1405
   [junit4]   2> 4765504 INFO  (zkCallback-7162-thread-1-processing-n:127.0.0.1:63227_) [n:127.0.0.1:63227_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 4765504 INFO  (zkCallback-7172-thread-1-processing-n:127.0.0.1:46006_) [n:127.0.0.1:46006_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 4765566 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-3-001 of type NRT
   [junit4]   2> 4765567 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 4765568 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5cbab22f{/,null,AVAILABLE}
   [junit4]   2> 4765568 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@160620c9{HTTP/1.1,[http/1.1]}{127.0.0.1:60593}
   [junit4]   2> 4765568 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.Server Started @4770182ms
   [junit4]   2> 4765568 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/, hostPort=60593, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-3-001/cores}
   [junit4]   2> 4765571 ERROR (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 4765571 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 4765571 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 4765571 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 4765571 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-07T00:38:02.431Z
   [junit4]   2> 4765575 INFO  (zkConnectionManagerCallback-7177-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4765577 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4765577 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-3-001/solr.xml
   [junit4]   2> 4765580 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 4765580 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 4765581 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 4765584 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51014/solr
   [junit4]   2> 4765586 INFO  (zkConnectionManagerCallback-7181-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4765590 INFO  (zkConnectionManagerCallback-7183-thread-1-processing-n:127.0.0.1:60593_) [n:127.0.0.1:60593_    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4765600 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:60593_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 4765604 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:60593_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4765608 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:60593_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60593_
   [junit4]   2> 4765610 INFO  (zkCallback-7162-thread-1-processing-n:127.0.0.1:63227_) [n:127.0.0.1:63227_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 4765610 INFO  (zkCallback-7154-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 4765610 INFO  (zkCallback-7144-thread-1-processing-n:127.0.0.1:33744_) [n:127.0.0.1:33744_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 4765610 INFO  (zkCallback-7172-thread-1-processing-n:127.0.0.1:46006_) [n:127.0.0.1:46006_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 4765613 INFO  (zkCallback-7182-thread-1-processing-n:127.0.0.1:60593_) [n:127.0.0.1:60593_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 4765809 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:60593_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1
   [junit4]   2> 4765816 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:60593_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1
   [junit4]   2> 4765817 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:60593_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1
   [junit4]   2> 4765818 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [n:127.0.0.1:60593_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-3-001/cores
   [junit4]   2> 4765852 INFO  (qtp215995676-34335) [n:127.0.0.1:33744_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:60593_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4765856 INFO  (OverseerCollectionConfigSetProcessor-99305748897595396-127.0.0.1:33744_-n_0000000000) [n:127.0.0.1:33744_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 4765857 INFO  (OverseerThreadFactory-12310-thread-5-processing-n:127.0.0.1:33744_) [n:127.0.0.1:33744_    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:60593_ for creating new replica
   [junit4]   2> 4765860 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 4765861 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 4765967 INFO  (zkCallback-7162-thread-1-processing-n:127.0.0.1:63227_) [n:127.0.0.1:63227_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 4765967 INFO  (zkCallback-7172-thread-1-processing-n:127.0.0.1:46006_) [n:127.0.0.1:46006_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 4765967 INFO  (zkCallback-7182-thread-1-processing-n:127.0.0.1:60593_) [n:127.0.0.1:60593_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 4766882 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 4766904 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.IndexSchema [collection1_shard2_replica_n45] Schema name=test
   [junit4]   2> 4766998 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 4767017 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n45' using configuration from collection collection1, trusted=true
   [junit4]   2> 4767018 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n45' (registry 'solr.core.collection1.shard2.replica_n45') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1
   [junit4]   2> 4767018 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 4767018 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrCore [[collection1_shard2_replica_n45] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-3-001/cores/collection1_shard2_replica_n45], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-3-001/cores/collection1_shard2_replica_n45/data/]
   [junit4]   2> 4767020 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=24, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 4767022 WARN  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 4767055 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 4767056 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 4767056 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 4767056 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 4767057 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=38.16796875, floorSegmentMB=1.3203125, forceMergeDeletesPctAllowed=7.580967010430552, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7145361062711567
   [junit4]   2> 4767057 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.SolrIndexSearcher Opening [Searcher@6b77024c[collection1_shard2_replica_n45] main]
   [junit4]   2> 4767059 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 4767060 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4767060 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 4767060 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588891991586897920
   [junit4]   2> 4767061 INFO  (searcherExecutor-12346-thread-1-processing-n:127.0.0.1:60593_ x:collection1_shard2_replica_n45 s:shard2 c:collection1) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrCore [collection1_shard2_replica_n45] Registered new searcher Searcher@6b77024c[collection1_shard2_replica_n45] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4767064 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.ZkController Core needs to recover:collection1_shard2_replica_n45
   [junit4]   2> 4767065 INFO  (updateExecutor-7178-thread-1-processing-n:127.0.0.1:60593_ x:collection1_shard2_replica_n45 s:shard2 c:collection1) [n:127.0.0.1:60593_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 4767065 INFO  (recoveryExecutor-7179-thread-1-processing-n:127.0.0.1:60593_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:60593_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 4767065 INFO  (recoveryExecutor-7179-thread-1-processing-n:127.0.0.1:60593_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:60593_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 4767065 INFO  (recoveryExecutor-7179-thread-1-processing-n:127.0.0.1:60593_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:60593_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard2_replica_n45]
   [junit4]   2> 4767065 INFO  (qtp1648888062-34442) [n:127.0.0.1:60593_ c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1205
   [junit4]   2> 4767065 INFO  (recoveryExecutor-7179-thread-1-processing-n:127.0.0.1:60593_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:60593_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 4767066 INFO  (recoveryExecutor-7179-thread-1-processing-n:127.0.0.1:60593_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:60593_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard2_replica_n45] as recovering, leader is [http://127.0.0.1:63227/collection1_shard2_replica_n41/] and I am [http://127.0.0.1:60593/collection1_shard2_replica_n45/]
   [junit4]   2> 4767068 INFO  (qtp215995676-34335) [n:127.0.0.1:33744_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:60593_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1216
   [junit4]   2> 4767070 INFO  (recoveryExecutor-7179-thread-1-processing-n:127.0.0.1:60593_ x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:60593_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:63227]; [WaitForState: action=PREPRECOVERY&core=collection1_shard2_replica_n41&nodeName=127.0.0.1:60593_&coreNodeName=core_node46&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 4767071 INFO  (qtp1850474082-34379) [n:127.0.0.1:63227_    ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node46, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
   [junit4]   2> 4767072 INFO  (qtp1850474082-34379) [n:127.0.0.1:63227_    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1_shard2_replica_n41, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:60593_, coreNodeName=core_node46, onlyIfActiveCheckResult=false, nodeProps: core_node46:{"core":"collection1_shard2_replica_n45","base_url":"http://127.0.0.1:60593","node_name":"127.0.0.1:60593_","state":"down","type":"NRT"}
   [junit4]   2> 4767169 INFO  (zkCallback-7172-thread-1-processing-n:127.0.0.1:46006_) [n:127.0.0.1:46006_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 4767169 INFO  (zkCallback-7182-thread-1-processing-n:127.0.0.1:60593_) [n:127.0.0.1:60593_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 4767169 INFO  (zkCallback-7162-thread-1-processing-n:127.0.0.1:63227_) [n:127.0.0.1:63227_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 4767187 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-4-001 of type NRT
   [junit4]   2> 4767187 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 4767188 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@76289c7b{/,null,AVAILABLE}
   [junit4]   2> 4767188 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@59ff7507{HTTP/1.1,[http/1.1]}{127.0.0.1:44102}
   [junit4]   2> 4767189 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.e.j.s.Server Started @4771802ms
   [junit4]   2> 4767189 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/tempDir-001/jetty4, solrconfig=solrconfig.xml, hostContext=/, hostPort=44102, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-4-001/cores}
   [junit4]   2> 4767191 ERROR (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 4767191 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 4767191 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 4767191 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 4767191 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-07T00:38:04.051Z
   [junit4]   2> 4767193 INFO  (zkConnectionManagerCallback-7188-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4767195 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4767195 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001/shard-4-001/solr.xml
   [junit4]   2> 4767197 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 4767197 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 4767198 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F7D6A333BE924370]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@47e5dff1, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 4767200 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[F

[...truncated too long message...]

m concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 7 /solr/clusterprops.json
   [junit4]   2> 7 /solr/clusterstate.json
   [junit4]   2> 5 /solr/autoscaling.json
   [junit4]   2> 4 /solr/collections/collection1/state.json
   [junit4]   2> 2 /solr/collections/.system/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 7 /solr/live_nodes
   [junit4]   2> 7 /solr/collections
   [junit4]   2> 5 /solr/overseer/queue
   [junit4]   2> 5 /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 5 /solr/overseer/collection-queue-work
   [junit4]   2> 5 /solr/overseer/queue-work
   [junit4]   2>
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDynamicLoading -Dtests.method=testDynamicLoading -Dtests.seed=F7D6A333BE924370 -Dtests.slow=true -Dtests.locale=sq-AL -Dtests.timezone=Etc/GMT+4 -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 28.0s J1 | TestDynamicLoading.testDynamicLoading <<<
   [junit4]    > Throwable #1: java.lang.AssertionError
   [junit4]    > at __randomizedtesting.SeedInfo.seed([F7D6A333BE924370:2F9B8E64494FE6D0]:0)
   [junit4]    > at org.apache.solr.core.TestDynamicLoading.testDynamicLoading(TestDynamicLoading.java:97)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading_F7D6A333BE924370-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=644, maxMBSortInHeap=5.434257561559814, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@32b0497a), locale=sq-AL, timezone=Etc/GMT+4
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=344547112,total=508088320
   [junit4]   2> NOTE: All tests run in this JVM: [DateRangeFieldTest, AliasIntegrationTest, TestRTimerTree, TestRandomFaceting, TolerantUpdateProcessorTest, TestCoreAdminApis, SolrMetricsIntegrationTest, TestBadConfig, TestCrossCoreJoin, DistributedQueryComponentCustomSortTest, TestReplicaProperties, TestCollationFieldDocValues, TestReloadAndDeleteDocs, TestGenericDistributedQueue, AssignTest, TestEmbeddedSolrServerConstructors, TestCollapseQParserPlugin, TermVectorComponentTest, MBeansHandlerTest, BlockJoinFacetRandomTest, SolrGraphiteReporterTest, TestPKIAuthenticationPlugin, CustomCollectionTest, TestZkAclsWithHadoopAuth, TestCloudManagedSchema, CdcrRequestHandlerTest, HdfsRecoverLeaseTest, TestInitQParser, EnumFieldTest, SystemInfoHandlerTest, TestRemoteStreaming, TestPartialUpdateDeduplication, LukeRequestHandlerTest, TestPayloadScoreQParserPlugin, ShardRoutingCustomTest, DistributedDebugComponentTest, TestFieldSortValues, SolrXmlInZkTest, SortSpecParsingTest, TestGroupingSearch, TestDynamicFieldResource, DirectSolrSpellCheckerTest, TestSolrConfigHandlerCloud, SynonymTokenizerTest, TestSolr4Spatial2, TestSlowCompositeReaderWrapper, PreAnalyzedFieldTest, PluginInfoTest, CloudExitableDirectoryReaderTest, JsonLoaderTest, TestRestManager, ZkStateReaderTest, TestSolrCloudWithSecureImpersonation, ClassificationUpdateProcessorTest, TestDocSet, TestNonDefinedSimilarityFactory, TestNumericTerms64, BJQParserTest, DocValuesMultiTest, StressHdfsTest, TestSolrFieldCacheBean, TestFoldingMultitermQuery, TestSystemIdResolver, DistributedQueueTest, StatsReloadRaceTest, RegexBytesRefFilterTest, TestDistributedStatsComponentCardinality, TestBlendedInfixSuggestions, CollectionStateFormat2Test, HdfsRestartWhileUpdatingTest, TestJavabinTupleStreamParser, TestChildDocTransformer, DebugComponentTest, TestReloadDeadlock, TestDocBasedVersionConstraints, FileUtilsTest, SolrCLIZkUtilsTest, TestJettySolrRunner, TestSolrCoreSnapshots, TestDistribIDF, TestClassicSimilarityFactory, DocumentAnalysisRequestHandlerTest, ReplicationFactorTest, TestTestInjection, SearchRateTriggerTest, SuggestComponentContextFilterQueryTest, TestCopyFieldCollectionResource, HdfsChaosMonkeyNothingIsSafeTest, GraphQueryTest, TestSchemaVersionResource, PeerSyncReplicationTest, TestFaceting, OverseerStatusTest, TestFunctionQuery, TestTriggerIntegration, HealthCheckHandlerTest, LeaderElectionIntegrationTest, DOMUtilTest, SpatialRPTFieldTypeTest, SaslZkACLProviderTest, SparseHLLTest, TestDistributedGrouping, ChaosMonkeyNothingIsSafeWithPullReplicasTest, TestTolerantUpdateProcessorRandomCloud, TestConfigSetsAPIZkFailure, OpenExchangeRatesOrgProviderTest, IndexSchemaRuntimeFieldTest, HdfsThreadLeakTest, ParsingFieldUpdateProcessorsTest, TestSha256AuthenticationProvider, TestReload, TestStressVersions, DeleteLastCustomShardedReplicaTest, DocExpirationUpdateProcessorFactoryTest, TestSolrDeletionPolicy1, TestSort, TestLuceneMatchVersion, TestMergePolicyConfig, TestStressUserVersions, SoftAutoCommitTest, TestUniqueKeyFieldResource, QueryEqualityTest, CollectionsAPIDistributedZkTest, TestSuggestSpellingConverter, TestTlogReplica, TestSimpleTextCodec, TestTrie, SpellPossibilityIteratorTest, SmileWriterTest, TestFieldTypeResource, HttpTriggerListenerTest, DistributedFacetExistsSmallTest, OverriddenZkACLAndCredentialsProvidersTest, TestCharFilters, BufferStoreTest, TestConfigReload, DistribDocExpirationUpdateProcessorTest, TestSolrQueryParser, HdfsBasicDistributedZk2Test, TestWordDelimiterFilterFactory, ShufflingReplicaListTransformerTest, DistributedSuggestComponentTest, TestRawResponseWriter, TestDistribStateManager, DistributedFacetPivotLongTailTest, CloudMLTQParserTest, TestHdfsCloudBackupRestore, VMParamsZkACLAndCredentialsProvidersTest, TestLeaderElectionZkExpiry, CursorMarkTest, TestSolrCloudWithDelegationTokens, SolrPluginUtilsTest, TestFastWriter, TestSystemCollAutoCreate, TestReqParamsAPI, OpenCloseCoreStressTest, ExecutePlanActionTest, TestSegmentSorting, BasicDistributedZk2Test, CdcrUpdateLogTest, MoveReplicaTest, SimpleFacetsTest, TestStressLucene, TestQueryTypes, CoreMergeIndexesAdminHandlerTest, TestLuceneIndexBackCompat, TestSimpleTrackingShardHandler, UpdateRequestProcessorFactoryTest, TestHashPartitioner, RollingRestartTest, TestLegacyFieldCache, ClassificationUpdateProcessorFactoryTest, BigEndianAscendingWordSerializerTest, FullSolrCloudDistribCmdsTest, TestRequestStatusCollectionAPI, TestLegacyField, DistanceUnitsTest, TestInPlaceUpdatesStandalone, TestPrepRecovery, FieldMutatingUpdateProcessorTest, MergeStrategyTest, ZkNodePropsTest, TestFieldCacheVsDocValues, VersionInfoTest, SolrIndexConfigTest, SearchHandlerTest, TestFieldCache, TestUseDocValuesAsStored, ChaosMonkeySafeLeaderTest, SyncSliceTest, BasicZkTest, RecoveryZkTest, ZkSolrClientTest, TestZkChroot, ZkCLITest, TestDistributedSearch, TestRecovery, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, ZkControllerTest, TestRealTimeGet, TestStressReorder, TestCoreContainer, SolrCoreTest, SolrCmdDistributorTest, QueryElevationComponentTest, PeerSyncTest, ConvertedLegacyTest, BasicFunctionalityTest, TestIndexSearcher, SolrIndexSplitterTest, SimplePostToolTest, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, CoreAdminHandlerTest, SuggesterTSTTest, SuggesterTest, SpatialFilterTest, TestCSVLoader, PolyFieldTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, DirectUpdateHandlerOptimizeTest, StatelessScriptUpdateProcessorFactoryTest, DisMaxRequestHandlerTest, TestOmitPositions, TermsComponentTest, IndexSchemaTest, ReturnFieldsTest, BinaryUpdateRequestHandlerTest, CSVRequestHandlerTest, TestComponentsName, SearchHandlerTest, HighlighterConfigTest, SOLR749Test, UpdateParamsTest, AlternateDirectoryTest, TestSolrIndexConfig, CopyFieldTest, ResponseLogComponentTest, BadComponentTest, SampleTest, TestSearchPerf, NumericFieldsTest, MinimalSchemaTest, TestElisionMultitermQuery, TestConfig, OutputWriterTest, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, DirectSolrConnectionTest, TestPhraseSuggestions, TestPerFieldSimilarity, TestBM25SimilarityFactory, TestFastOutputStream, TestFastLRUCache, DateMathParserTest, ChaosMonkeyNothingIsSafeTest, CollectionsAPIAsyncDistributedZkTest, CreateCollectionCleanupTest, DeleteShardTest, DeleteStatusTest, DocValuesNotIndexedTest, ForceLeaderTest, LeaderInitiatedRecoveryOnShardRestartTest, LegacyCloudClusterPropTest, MigrateRouteKeyTest, MoveReplicaHDFSTest, NodeMutatorTest, OverseerCollectionConfigSetProcessorTest, OverseerModifyCollectionTest, OverseerTaskQueueTest, SharedFSAutoReplicaFailoverTest, SolrCloudExampleTest, TestAuthenticationFramework, TestCloudDeleteByQuery, TestCloudInspectUtil, TestCloudJSONFacetJoinDomain, TestCloudPivotFacet, TestCloudPseudoReturnFields, TestCloudRecovery, TestCloudSearcherWarming, TestCollectionAPI, TestCollectionsAPIViaSolrCloudCluster, TestConfigSetsAPI, TestDownShardTolerantSearch, TestLeaderInitiatedRecoveryThread, TestLocalFSCloudBackupRestore, TestOnReconnectListenerSupport, TestPullReplica, TestSSLRandomization, TestSizeLimitedDistributedMap, TestTolerantUpdateProcessorCloud, TestUtilizeNode, TlogReplayBufferedWhileIndexingTest, ZkFailoverTest, AutoScalingHandlerTest, NodeAddedTriggerTest, NodeLostTriggerTest, SystemLogListenerTest, TestPolicyCloud, TestExecutePlanAction, TestLargeCluster, ImplicitSnitchTest, RuleEngineTest, RulesTest, CachingDirectoryFactoryTest, ConfigureRecoveryStrategyTest, DirectoryFactoryTest, HdfsDirectoryFactoryTest, TestDynamicLoading]
   [junit4] Completed [620/766 (1!)] on J1 in 28.04s, 1 test, 1 failure <<< FAILURES!

[...truncated 48523 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1611 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1611/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:     1) Thread[id=37096, name=jetty-launcher-10176-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
   1) Thread[id=37096, name=jetty-launcher-10176-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
        at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
        at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
        at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
        at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
        at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
        at __randomizedtesting.SeedInfo.seed([6C9B73DD0AE286F9]:0)




Build Log:
[...truncated 13505 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_6C9B73DD0AE286F9-001/init-core-data-001
   [junit4]   2> 4107256 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[6C9B73DD0AE286F9]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 4107257 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[6C9B73DD0AE286F9]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 4107257 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[6C9B73DD0AE286F9]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 4107295 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[6C9B73DD0AE286F9]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_6C9B73DD0AE286F9-001/tempDir-001
   [junit4]   2> 4107295 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[6C9B73DD0AE286F9]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 4107295 INFO  (Thread-7029) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 4107295 INFO  (Thread-7029) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 4107302 ERROR (Thread-7029) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 4107396 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[6C9B73DD0AE286F9]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:36311
   [junit4]   2> 4107402 INFO  (zkConnectionManagerCallback-10179-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4107408 INFO  (jetty-launcher-10176-thread-1) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 4107408 INFO  (jetty-launcher-10176-thread-2) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 4107411 INFO  (jetty-launcher-10176-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@746b9c7c{/solr,null,AVAILABLE}
   [junit4]   2> 4107411 INFO  (jetty-launcher-10176-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@372d4f9a{/solr,null,AVAILABLE}
   [junit4]   2> 4107418 INFO  (jetty-launcher-10176-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@322a347c{HTTP/1.1,[http/1.1]}{127.0.0.1:37013}
   [junit4]   2> 4107418 INFO  (jetty-launcher-10176-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@75b8dc9c{HTTP/1.1,[http/1.1]}{127.0.0.1:33275}
   [junit4]   2> 4107418 INFO  (jetty-launcher-10176-thread-2) [    ] o.e.j.s.Server Started @4111904ms
   [junit4]   2> 4107418 INFO  (jetty-launcher-10176-thread-1) [    ] o.e.j.s.Server Started @4111904ms
   [junit4]   2> 4107418 INFO  (jetty-launcher-10176-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33275}
   [junit4]   2> 4107418 INFO  (jetty-launcher-10176-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37013}
   [junit4]   2> 4107418 ERROR (jetty-launcher-10176-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 4107418 ERROR (jetty-launcher-10176-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 4107418 INFO  (jetty-launcher-10176-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 4107418 INFO  (jetty-launcher-10176-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 4107418 INFO  (jetty-launcher-10176-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 4107418 INFO  (jetty-launcher-10176-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 4107418 INFO  (jetty-launcher-10176-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 4107419 INFO  (jetty-launcher-10176-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-07T10:36:27.934Z
   [junit4]   2> 4107418 INFO  (jetty-launcher-10176-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 4107419 INFO  (jetty-launcher-10176-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-07T10:36:27.934Z
   [junit4]   2> 4107421 INFO  (zkConnectionManagerCallback-10181-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4107421 INFO  (zkConnectionManagerCallback-10183-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4107422 INFO  (jetty-launcher-10176-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 4107422 INFO  (jetty-launcher-10176-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 4107430 INFO  (jetty-launcher-10176-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36311/solr
   [junit4]   2> 4107430 INFO  (jetty-launcher-10176-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36311/solr
   [junit4]   2> 4107432 INFO  (zkConnectionManagerCallback-10189-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4107432 INFO  (zkConnectionManagerCallback-10191-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4107435 INFO  (zkConnectionManagerCallback-10193-thread-1-processing-n:127.0.0.1:37013_solr) [n:127.0.0.1:37013_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4107435 INFO  (zkConnectionManagerCallback-10195-thread-1-processing-n:127.0.0.1:33275_solr) [n:127.0.0.1:33275_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4107575 INFO  (jetty-launcher-10176-thread-2) [n:127.0.0.1:33275_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4107575 INFO  (jetty-launcher-10176-thread-1) [n:127.0.0.1:37013_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4107576 INFO  (jetty-launcher-10176-thread-2) [n:127.0.0.1:33275_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33275_solr
   [junit4]   2> 4107577 INFO  (jetty-launcher-10176-thread-1) [n:127.0.0.1:37013_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37013_solr
   [junit4]   2> 4107577 INFO  (jetty-launcher-10176-thread-2) [n:127.0.0.1:33275_solr    ] o.a.s.c.Overseer Overseer (id=99308102471778310-127.0.0.1:33275_solr-n_0000000000) starting
   [junit4]   2> 4107581 INFO  (zkCallback-10194-thread-1-processing-n:127.0.0.1:33275_solr) [n:127.0.0.1:33275_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 4107581 INFO  (zkCallback-10192-thread-1-processing-n:127.0.0.1:37013_solr) [n:127.0.0.1:37013_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 4107590 INFO  (jetty-launcher-10176-thread-2) [n:127.0.0.1:33275_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33275_solr
   [junit4]   2> 4107592 INFO  (zkCallback-10192-thread-1-processing-n:127.0.0.1:37013_solr) [n:127.0.0.1:37013_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 4107592 INFO  (zkCallback-10194-thread-1-processing-n:127.0.0.1:33275_solr) [n:127.0.0.1:33275_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 4107615 INFO  (jetty-launcher-10176-thread-2) [n:127.0.0.1:33275_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
   [junit4]   2> 4107615 WARN  (jetty-launcher-10176-thread-2) [n:127.0.0.1:33275_solr    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 4107615 INFO  (jetty-launcher-10176-thread-2) [n:127.0.0.1:33275_solr    ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
   [junit4]   2> 4107617 INFO  (jetty-launcher-10176-thread-2) [n:127.0.0.1:33275_solr    ] o.a.c.f.i.CuratorFrameworkImpl Starting
   [junit4]   2> 4107631 INFO  (jetty-launcher-10176-thread-2-EventThread) [n:127.0.0.1:33275_solr    ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
   [junit4]   2> 4107633 INFO  (jetty-launcher-10176-thread-1) [n:127.0.0.1:37013_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
   [junit4]   2> 4107634 WARN  (jetty-launcher-10176-thread-1) [n:127.0.0.1:37013_solr    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 4107634 INFO  (jetty-launcher-10176-thread-1) [n:127.0.0.1:37013_solr    ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
   [junit4]   2> 4107635 INFO  (jetty-launcher-10176-thread-1) [n:127.0.0.1:37013_solr    ] o.a.c.f.i.CuratorFrameworkImpl Starting
   [junit4]   2> 4107637 INFO  (jetty-launcher-10176-thread-1-EventThread) [n:127.0.0.1:37013_solr    ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
   [junit4]   2> 4107658 WARN  (jetty-launcher-10176-thread-2) [n:127.0.0.1:33275_solr    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 4107658 WARN  (jetty-launcher-10176-thread-1) [n:127.0.0.1:37013_solr    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 4107720 INFO  (jetty-launcher-10176-thread-2) [n:127.0.0.1:33275_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33275.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2466aef
   [junit4]   2> 4107720 INFO  (jetty-launcher-10176-thread-1) [n:127.0.0.1:37013_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37013.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2466aef
   [junit4]   2> 4107725 INFO  (jetty-launcher-10176-thread-2) [n:127.0.0.1:33275_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33275.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2466aef
   [junit4]   2> 4107726 INFO  (jetty-launcher-10176-thread-2) [n:127.0.0.1:33275_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33275.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2466aef
   [junit4]   2> 4107726 INFO  (jetty-launcher-10176-thread-1) [n:127.0.0.1:37013_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37013.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2466aef
   [junit4]   2> 4107726 INFO  (jetty-launcher-10176-thread-1) [n:127.0.0.1:37013_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37013.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2466aef
   [junit4]   2> 4107727 INFO  (jetty-launcher-10176-thread-2) [n:127.0.0.1:33275_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_6C9B73DD0AE286F9-001/tempDir-001/node2/.
   [junit4]   2> 4107727 INFO  (jetty-launcher-10176-thread-1) [n:127.0.0.1:37013_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_6C9B73DD0AE286F9-001/tempDir-001/node1/.
   [junit4]   2> 4107776 INFO  (zkConnectionManagerCallback-10201-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4107982 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNullProxyUser
   [junit4]   2> 4107991 INFO  (qtp1186287657-37043) [    ] o.a.s.s.SolrDispatchFilter Error authenticating
   [junit4]   2> java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
   [junit4]   2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
   [junit4]   2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
   [junit4]   2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
   [junit4]   2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
   [junit4]   2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
   [junit4]   2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
   [junit4]   2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
   [junit4]   2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
   [junit4]   2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
   [junit4]   2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:451)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
   [junit4]   2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
   [junit4]   2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 4107992 WARN  (qtp1186287657-37043) [    ] o.e.j.s.ServletHandler /solr/admin/collections
   [junit4]   2> org.apache.solr.common.SolrException: Error during request authentication,
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:457)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
   [junit4]   2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
   [junit4]   2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
   [junit4]   2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
   [junit4]   2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
   [junit4]   2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
   [junit4]   2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
   [junit4]   2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
   [junit4]   2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
   [junit4]   2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
   [junit4]   2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
   [junit4]   2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
   [junit4]   2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:451)
   [junit4]   2> ... 26 more
   [junit4]   2> 4107993 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNullProxyUser
   [junit4]   2> 4108188 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidGroup
   [junit4]   2> 4108229 WARN  (qtp1186287657-37048) [    ] o.a.h.s.ShellBasedUnixGroupsMapping got exception trying to get groups for user bar: id: invalid user name: "bar"
   [junit4]   2> id: invalid user name: "bar"
   [junit4]   2>
   [junit4]   2> 4108233 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[6C9B73DD0AE286F9]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bogusGroup&p=bogusGroup&t=test&e=1515357388705&s=czt8ZeIqD50g1ETURlAnKUz4fD0="; Path=/; Domain=127.0.0.1; Expires=dom, 07-ene-2018 20:36:28 GMT; HttpOnly". Invalid 'expires' attribute: dom, 07-ene-2018 20:36:28 GMT
   [junit4]   2> 4108233 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidGroup
   [junit4]   2> 4108436 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateHost
   [junit4]   2> 4108441 INFO  (qtp1186287657-37049) [n:127.0.0.1:37013_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4108441 INFO  (qtp1186287657-37049) [n:127.0.0.1:37013_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 4108441 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[6C9B73DD0AE286F9]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1515357388952&s=h0JB/Yep6tsZ9LxovyMJ1lBrpTQ="; Path=/; Domain=127.0.0.1; Expires=dom, 07-ene-2018 20:36:28 GMT; HttpOnly". Invalid 'expires' attribute: dom, 07-ene-2018 20:36:28 GMT
   [junit4]   2> 4108442 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateHost
   [junit4]   2> 4108642 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyUnknownRemote
   [junit4]   2> 4124496 WARN  (jetty-launcher-10176-thread-1-SendThread(127.0.0.1:36311)) [n:127.0.0.1:37013_solr    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 16805ms for sessionid 0x160d0316b260008
   [junit4]   2> 4124496 WARN  (jetty-launcher-10176-thread-1-SendThread(127.0.0.1:36311)) [n:127.0.0.1:37013_solr    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 16768ms for sessionid 0x160d0316b260005
   [junit4]   2> 4124496 WARN  (jetty-launcher-10176-thread-2-SendThread(127.0.0.1:36311)) [n:127.0.0.1:33275_solr    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 16769ms for sessionid 0x160d0316b260006
   [junit4]   2> 4124499 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x160d0316b260008, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 4124499 WARN  (jetty-launcher-10176-thread-2-SendThread(127.0.0.1:36311)) [n:127.0.0.1:33275_solr    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 16804ms for sessionid 0x160d0316b260007
   [junit4]   2> 4124503 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x160d0316b260005, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 4124513 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x160d0316b260007, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 4124513 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x160d0316b260006, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 4124521 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[6C9B73DD0AE286F9]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1515357389158&s=oWlBb49A7DBScfcPR82dUaePtzQ="; Path=/; Domain=127.0.0.1; Expires=dom, 07-ene-2018 20:36:29 GMT; HttpOnly". Invalid 'expires' attribute: dom, 07-ene-2018 20:36:29 GMT
   [junit4]   2> 4124521 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyUnknownRemote
   [junit4]   2> 4124606 INFO  (jetty-launcher-10176-thread-2-EventThread) [n:127.0.0.1:33275_solr    ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
   [junit4]   2> 4124612 WARN  (zkConnectionManagerCallback-10193-thread-1-processing-n:127.0.0.1:37013_solr) [n:127.0.0.1:37013_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5c52f6e2 name: ZooKeeperConnection Watcher:127.0.0.1:36311/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 4124616 WARN  (zkConnectionManagerCallback-10193-thread-1-processing-n:127.0.0.1:37013_solr) [n:127.0.0.1:37013_solr    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 4124618 WARN  (zkConnectionManagerCallback-10195-thread-1-processing-n:127.0.0.1:33275_solr) [n:127.0.0.1:33275_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6f0a5059 name: ZooKeeperConnection Watcher:127.0.0.1:36311/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 4124619 WARN  (zkConnectionManagerCallback-10195-thread-1-processing-n:127.0.0.1:33275_solr) [n:127.0.0.1:33275_solr    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 4124787 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidRemote
   [junit4]   2> 4124798 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[6C9B73DD0AE286F9]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1515357405302&s=ljfrd8qdoUad1Z1r6AlFb8By43g="; Path=/; Domain=127.0.0.1; Expires=dom, 07-ene-2018 20:36:45 GMT; HttpOnly". Invalid 'expires' attribute: dom, 07-ene-2018 20:36:45 GMT
   [junit4]   2> 4124798 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidRemote
   [junit4]   2> 4125170 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateAnyHostAnyUser
   [junit4]   2> 4125170 INFO  (qtp1186287657-37043) [n:127.0.0.1:37013_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4125171 INFO  (qtp1186287657-37043) [n:127.0.0.1:37013_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 4125171 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[6C9B73DD0AE286F9]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostAnyUser&p=anyHostAnyUser&t=test&e=1515357405685&s=SAzBERfdilXgXNnol6uPWiT8rXw="; Path=/; Domain=127.0.0.1; Expires=dom, 07-ene-2018 20:36:45 GMT; HttpOnly". Invalid 'expires' attribute: dom, 07-ene-2018 20:36:45 GMT
   [junit4]   2> 4125171 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateAnyHostAnyUser
   [junit4]   2> 4125497 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateGroup
   [junit4]   2> 4125532 INFO  (qtp1186287657-37046) [n:127.0.0.1:37013_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=jenkins&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4125533 INFO  (qtp1186287657-37046) [n:127.0.0.1:37013_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=jenkins&action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 4125533 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[6C9B73DD0AE286F9]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostUsersGroup&p=anyHostUsersGroup&t=test&e=1515357406013&s=YLxOx3ZkcHmdaM7GDVQOoTIICbQ="; Path=/; Domain=127.0.0.1; Expires=dom, 07-ene-2018 20:36:46 GMT; HttpOnly". Invalid 'expires' attribute: dom, 07-ene-2018 20:36:46 GMT
   [junit4]   2> 4125533 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateGroup
   [junit4]   2> 4125797 INFO  (zkConnectionManagerCallback-10193-thread-1-processing-n:127.0.0.1:37013_solr) [n:127.0.0.1:37013_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4125866 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigGroups
   [junit4]   2> 4125868 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[6C9B73DD0AE286F9]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noGroups&p=noGroups&t=test&e=1515357406382&s=l7mQJXpa9r8gB2GBB2Kd+cE+80Y="; Path=/; Domain=127.0.0.1; Expires=dom, 07-ene-2018 20:36:46 GMT; HttpOnly". Invalid 'expires' attribute: dom, 07-ene-2018 20:36:46 GMT
   [junit4]   2> 4125869 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigGroups
   [junit4]   2> 4126079 INFO  (jetty-launcher-10176-thread-1-EventThread) [n:127.0.0.1:37013_solr    ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
   [junit4]   2> 4126342 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyWrongHost
   [junit4]   2> 4127377 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[6C9B73DD0AE286F9]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=wrongHost&p=wrongHost&t=test&e=1515357406857&s=MYrCr9W8VXABitJfv2IntRDrheM="; Path=/; Domain=127.0.0.1; Expires=dom, 07-ene-2018 20:36:46 GMT; HttpOnly". Invalid 'expires' attribute: dom, 07-ene-2018 20:36:46 GMT
   [junit4]   2> 4127378 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyWrongHost
   [junit4]   2> 4127378 INFO  (zkConnectionManagerCallback-10195-thread-1-processing-n:127.0.0.1:33275_solr) [n:127.0.0.1:33275_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4127649 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testForwarding
   [junit4]   2> 4127652 INFO  (zkConnectionManagerCallback-10204-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4128855 INFO  (zkConnectionManagerCallback-10207-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 4128858 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 4128865 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36311/solr ready
   [junit4]   2> 4128876 INFO  (qtp256124474-37050) [n:127.0.0.1:33275_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4128891 INFO  (OverseerThreadFactory-14029-thread-1-processing-n:127.0.0.1:33275_solr) [n:127.0.0.1:33275_solr    ] o.a.s.c.CreateCollectionCmd Create collection forwardingCollection
   [junit4]   2> 4129010 INFO  (OverseerStateUpdate-99308102471778310-127.0.0.1:33275_solr-n_0000000000) [n:127.0.0.1:33275_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"forwardingCollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"forwardingCollection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:33275/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 4129223 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 4129223 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 4129336 INFO  (zkCallback-10194-thread-3-processing-n:127.0.0.1:33275_solr) [n:127.0.0.1:33275_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 4129338 INFO  (zkCallback-10194-thread-7-processing-n:127.0.0.1:33275_solr) [n:127.0.0.1:33275_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 4130250 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 4130276 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema [forwardingCollection_shard1_replica_n1] Schema name=test
   [junit4]   2> 4130464 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 4130542 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'forwardingCollection_shard1_replica_n1' using configuration from collection forwardingCollection, trusted=true
   [junit4]   2> 4130543 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33275.solr.core.forwardingCollection.shard1.replica_n1' (registry 'solr.core.forwardingCollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2466aef
   [junit4]   2> 4130544 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 4130544 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [[forwardingCollection_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_6C9B73DD0AE286F9-001/tempDir-001/node2/forwardingCollection_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_6C9B73DD0AE286F9-001/tempDir-001/node2/./forwardingCollection_shard1_replica_n1/data/]
   [junit4]   2> 4130551 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=24, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.10943892908552116]
   [junit4]   2> 4130640 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 4130640 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 4130643 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 4130643 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 4130644 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=33, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 4130644 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1e523cb5[forwardingCollection_shard1_replica_n1] main]
   [junit4]   2> 4130647 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 4130655 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 4130678 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 4130679 INFO  (searcherExecutor-14034-thread-1-processing-n:127.0.0.1:33275_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 4130679 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588929664064159744
   [junit4]   2> 4130712 INFO  (searcherExecutor-14034-thread-1-processing-n:127.0.0.1:33275_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 4130712 INFO  (searcherExecutor-14034-thread-1-processing-n:127.0.0.1:33275_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 4130713 INFO  (searcherExecutor-14034-thread-1-processing-n:127.0.0.1:33275_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 4130714 INFO  (searcherExecutor-14034-thread-1-processing-n:127.0.0.1:33275_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 4130714 INFO  (searcherExecutor-14034-thread-1-processing-n:127.0.0.1:33275_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 4130715 INFO  (searcherExecutor-14034-thread-1-processing-n:127.0.0.1:33275_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 4130715 INFO  (searcherExecutor-14034-thread-1-processing-n:127.0.0.1:33275_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 4130716 INFO  (searcherExecutor-14034-thread-1-processing-n:127.0.0.1:33275_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 4130717 INFO  (searcherExecutor-14034-thread-1-processing-n:127.0.0.1:33275_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] Registered new searcher Searcher@1e523cb5[forwardingCollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4130720 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 4130720 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 4130720 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33275/solr/forwardingCollection_shard1_replica_n1/
   [junit4]   2> 4130720 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 4130720 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:33275/solr/forwardingCollection_shard1_replica_n1/ has no replicas
   [junit4]   2> 4130720 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 4130729 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33275/solr/forwardingCollection_shard1_replica_n1/ shard1
   [junit4]   2> 4130832 INFO  (zkCallback-10194-thread-3-processing-n:127.0.0.1:33275_solr) [n:127.0.0.1:33275_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 4130832 INFO  (zkCallback-10194-thread-7-processing-n:127.0.0.1:33275_solr) [n:127.0.0.1:33275_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 4130881 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 4130884 INFO  (qtp256124474-37054) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1661
   [junit4]   2> 4130884 WARN  (httpShardExecutor-14023-thread-1-processing-n:127.0.0.1:33275_solr [http:////127.0.0.1:33275//solr] http:////127.0.0.1:33275//solr) [n:127.0.0.1:33275_solr    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1515357409734&s=gRSj9BvZe0fJvqyA/BYTDRPRaS8="; Path=/; Domain=127.0.0.1; Expires=dom, 07-ene-2018 20:36:49 GMT; HttpOnly". Invalid 'expires' attribute: dom, 07-ene-2018 20:36:49 GMT
   [junit4]   2> 4130890 INFO  (qtp256124474-37050) [n:127.0.0.1:33275_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 4130892 INFO  (OverseerCollectionConfigSetProcessor-99308102471778310-127.0.0.1:33275_solr-n_0000000000) [n:127.0.0.1:33275_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 4130984 INFO  (zkCallback-10194-thread-7-processing-n:127.0.0.1:33275_solr) [n:127.0.0.1:33275_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 4130984 INFO  (zkCallback-10194-thread-8-processing-n:127.0.0.1:33275_solr) [n:127.0.0.1:33275_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 4131891 INFO  (qtp256124474-37050) [n:127.0.0.1:33275_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=3015
   [junit4]   2> 4131891 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[6C9B73DD0AE286F9]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1515357409391&s=pWIctn7Lq+Gq1R0zCrwBXSIzfcw="; Path=/; Domain=127.0.0.1; Expires=dom, 07-ene-2018 20:36:49 GMT; HttpOnly". Invalid 'expires' attribute: dom, 07-ene-2018 20:36:49 GMT
   [junit4]   2> 4131892 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: forwardingCollection failOnTimeout:true timeout (sec):100
   [junit4]   2> 4131893 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: forwardingCollection
   [junit4]   2> 4131895 INFO  (qtp1186287657-37049) [n:127.0.0.1:37013_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 4131899 INFO  (qtp256124474-37056) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 4131899 WARN  (qtp1186287657-37049) [n:127.0.0.1:37013_solr    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1515357412413&s=VbXpGdPPAhtQ54A5oBT9yiOatkQ="; Path=/; Domain=127.0.0.1; Expires=dom, 07-ene-2018 20:36:52 GMT; HttpOnly". Invalid 'expires' attribute: dom, 07-ene-2018 20:36:52 GMT
   [junit4]   2> 4131900 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[6C9B73DD0AE286F9]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1515357412410&s=yKSowu63Fg4nTPE05Hzn9RA86eo="; Path=/; Domain=127.0.0.1; Expires=dom, 07-ene-2018 20:36:52 GMT; HttpOnly". Invalid 'expires' attribute: dom, 07-ene-2018 20:36:52 GMT
   [junit4]   2> 4131900 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[6C9B73DD0AE286F9]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1515357412413&s=VbXpGdPPAhtQ54A5oBT9yiOatkQ="; Path=/; Domain=127.0.0.1; Expires=dom, 07-ene-2018 20:36:52 GMT; HttpOnly". Invalid 'expires' attribute: dom, 07-ene-2018 20:36:52 GMT
   [junit4]   2> 4131903 INFO  (qtp256124474-37057) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 4131904 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[6C9B73DD0AE286F9]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1515357412417&s=bVgPj6/jQH8ajzTXTP1xjf2JHzA="; Path=/; Domain=127.0.0.1; Expires=dom, 07-ene-2018 20:36:52 GMT; HttpOnly". Invalid 'expires' attribute: dom, 07-ene-2018 20:36:52 GMT
   [junit4]   2> 4131904 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testForwarding
   [junit4]   2> 4132287 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigHosts
   [junit4]   2> 4132294 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[6C9B73DD0AE286F9]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noHosts&p=noHosts&t=test&e=1515357412806&s=pLov2M0C34R7QRS3+6jyeovnTV8="; Path=/; Domain=127.0.0.1; Expires=dom, 07-ene-2018 20:36:52 GMT; HttpOnly". Invalid 'expires' attribute: dom, 07-ene-2018 20:36:52 GMT
   [junit4]   2> 4132294 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigHosts
   [junit4]   2> 4132625 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidProxyUser
   [junit4]   2> 4132628 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[6C9B73DD0AE286F9]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bar&p=bar&t=test&e=1515357413141&s=mJmATWCGFp68BGCPn2vM8LP1kYg="; Path=/; Domain=127.0.0.1; Expires=dom, 07-ene-2018 20:36:53 GMT; HttpOnly". Invalid 'expires' attribute: dom, 07-ene-2018 20:36:53 GMT
   [junit4]   2> 4132628 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[6C9B73DD0AE286F9]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidProxyUser
   [junit4]   2> 4132631 INFO  (jetty-closer-10177-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@322a347c{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 4132635 INFO  (jetty-closer-10177-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=948913322
   [junit4]   2> 4132636 INFO  (jetty-closer-10177-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 4132636 INFO  (jetty-closer-10177-thread-2) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@75b8dc9c{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 4132636 INFO  (jetty-closer-10177-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3dd242a7: rootName = solr_37013, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@3d0ad231
   [junit4]   2> 4132640 INFO  (jetty-closer-10177-thread-2) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=339249551
   [junit4]   2> 4132645 INFO  (jetty-closer-10177-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 4132645 INFO  (jetty-closer-10177-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@47b3fbe: rootName = solr_33275, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@2ba081cd
   [junit4]   2> 4132649 INFO  (jetty-closer-10177-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 4132649 INFO  (jetty-closer-10177-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@66be3c7: rootName = solr_37013, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@7c3171e8
   [junit4]   2> 4132656 INFO  (jetty-closer-10177-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 4132656 INFO  (jetty-closer-10177-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@76dcf692: rootName = solr_33275, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@7c3171e8
   [junit4]   2> 4132658 INFO  (jetty-closer-10177-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 4132658 INFO  (jetty-closer-10177-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5d16ae9f: rootName = solr_37013, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2a2c0689
   [junit4]   2> 4132661 INFO  (jetty-closer-10177-thread-1) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37013_solr
   [junit4]   2> 4132661 INFO  (jetty-closer-10177-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 4132661 INFO  (jetty-closer-10177-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2e842d95: rootName = solr_33275, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2a2c0689
   [junit4]   2> 4132662 INFO  (jetty-closer-10177-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 4132662 INFO  (zkCallback-10192-thread-1-processing-n:127.0.0.1:37013_solr) [n:127.0.0.1:37013_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 4132663 INFO  (zkCallback-10194-thread-8-processing-n:127.0.0.1:33275_solr) [n:127.0.0.1:33275_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 4132669 INFO  (jetty-closer-10177-thread-2) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:33275_solr
   [junit4]   2> 4132670 INFO  (jetty-closer-10177-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 4132670 ERROR (Thread[Thread-7048,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:37013_solr    ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 4132671 INFO  (zkCallback-10194-thread-8-processing-n:127.0.0.1:33275_solr) [n:127.0.0.1:33275_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 4132671 INFO  (coreCloseExecutor-14040-thread-1) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@59022425
   [junit4]   2> 4132672 INFO  (coreCloseExecutor-14040-thread-1) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.forwardingCollection.shard1.replica_n1, tag=1493312549
   [junit4]   2> 4132673 INFO  (coreCloseExecutor-14040-thread-1) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7cf7eadd: rootName = solr_33275, domain = solr.core.forwardingCollection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.forwardingCollection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@7f671d
   [junit4]   2> 4132688 INFO  (coreCloseExecutor-14040-thread-1) [n:127.0.0.1:33275_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.forwardingCollection.shard1.leader, tag=1493312549
   [junit4]   2> 4132694 INFO  (jetty-closer-10177-thread-2) [    ] o.a.s.c.Overseer Overseer (id=99308102471778310-127.0.0.1:33275_solr-n_0000000000) closing
   [junit4]   2> 4132695 INFO  (OverseerStateUpdate-99308102471778310-127.0.0.1:33275_solr-n_0000000000) [n:127.0.0.1:33275_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:33275_solr
   [junit4]   2> 4132696 WARN  (OverseerAutoScalingTriggerThread-99308102471778310-127.0.0.1:33275_solr-n_0000000000) [n:127.0.0.1:33275_solr    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 4132696 INFO  (jetty-closer-10177-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@746b9c7c{/solr,null,UNAVAILABLE}
   [junit4]   2> 4132714 ERROR (Thread[Thread-7046,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:33275_solr    ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 4132724 INFO  (jetty-closer-10177-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@372d4f9a{/solr,null,UNAVAILABLE}
   [junit4]   2> 4132731 ERROR (SUITE-TestSolrCloudWithSecureImpersonation-seed#[6C9B73DD0AE286F9]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 4132731 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[6C9B73DD0AE286F9]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36311 36311
   [junit4]   2> 4134589 INFO  (Thread-7029) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36311 36311
   [junit4]   2> 4134601 WARN  (Thread-7029) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/aliases.json
   [junit4]   2> 3 /solr/clusterprops.json
   [junit4]   2> 2 /solr/security.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 5 /solr/collections/forwardingCollection/state.json
   [junit4]   2> 4 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMKeyIdRoot
   [junit4]   2> 3 /solr/clusterstate.json
   [junit4]   2> 3 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMSeqNumRoot
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_1
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_3
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_2
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_4
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/live_nodes
   [junit4]   2> 3 /solr/collections
   [junit4]   2> 3 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot
   [junit4]   2> 3 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot
   [junit4]   2>
   [junit4]   2> 4134633 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[6C9B73DD0AE286F9]-worker) [    ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
   [junit4]   2> Jan 07, 2018 10:36:55 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> 4154685 ERROR (jetty-launcher-10176-thread-2-EventThread) [n:127.0.0.1:33275_solr    ] o.a.c.f.i.CuratorFrameworkImpl Watcher exception
   [junit4]   2> java.lang.IllegalStateException: Client is not started
   [junit4]   2> at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
   [junit4]   2> at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:113)
   [junit4]   2> at org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:477)
   [junit4]   2> at org.apache.curator.framework.imps.NamespaceFacade.getZooKeeper(NamespaceFacade.java:112)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:302)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:291)
   [junit4]   2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
   [junit4]   2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
   [junit4]   2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
   [junit4]   2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
   [junit4]   2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> Jan 07, 2018 10:37:15 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
   [junit4]   2>    1) Thread[id=37096, name=jetty-launcher-10176-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
   [junit4]   2>         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
   [junit4]   2>         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
   [junit4]   2>         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
   [junit4]   2>         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
   [junit4]   2>         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
   [junit4]   2>         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
   [junit4]   2>         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
   [junit4]   2>         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
   [junit4]   2>         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
   [junit4]   2>         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
   [junit4]   2>         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2>         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> Jan 07, 2018 10:37:15 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=37096, name=jetty-launcher-10176-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
   [junit4]   2> 4154921 ERROR (jetty-launcher-10176-thread-1-EventThread) [n:127.0.0.1:37013_solr    ] o.a.c.f.i.CuratorFrameworkImpl Watcher exception
   [junit4]   2> java.lang.InterruptedException
   [junit4]   2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
   [junit4]   2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
   [junit4]   2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
   [junit4]   2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
   [junit4]   2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
   [junit4]   2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
   [junit4]   2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
   [junit4]   2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
   [junit4]   2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> Jan 07, 2018 10:37:15 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@139f3644), locale=es-DO, timezone=America/Indiana/Indianapolis
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=248821392,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [TestPartialUpdateDeduplication, UpdateParamsTest, TestLegacyField, AssignTest, LukeRequestHandlerTest, HdfsThreadLeakTest, TestSlowCompositeReaderWrapper, TestSolrFieldCacheBean, DocValuesNotIndexedTest, TestSubQueryTransformerCrossCore, TestCrossCoreJoin, TestDynamicFieldResource, TestDocBasedVersionConstraints, CollectionsAPIDistributedZkTest, TestConfig, TestSolrCloudWithDelegationTokens, BasicZkTest, SearchRateTriggerTest, CollectionsAPIAsyncDistributedZkTest, SystemInfoHandlerTest, TestJsonFacets, NodeMutatorTest, TestStressInPlaceUpdates, SimplePostToolTest, ClassificationUpdateProcessorFactoryTest, WrapperMergePolicyFactoryTest, AutoScalingHandlerTest, DistributedIntervalFacetingTest, SharedFSAutoReplicaFailoverTest, TestNodeAddedTrigger, OpenCloseCoreStressTest, TestLRUCache, DistanceUnitsTest, TestBlendedInfixSuggestions, BasicDistributedZk2Test, TestCloudSearcherWarming, EnumFieldTest, TestStressLucene, TestHdfsCloudBackupRestore, TestScoreJoinQPScore, TestSQLHandler, SaslZkACLProviderTest, PeerSyncTest, BadCopyFieldTest, LeaderElectionIntegrationTest, DistributedMLTComponentTest, TestFaceting, TestManagedResource, TestRandomRequestDistribution, DistributedFacetSimpleRefinementLongTailTest, TestPostingsSolrHighlighter, InfixSuggestersTest, SpatialRPTFieldTypeTest, TestCloudPseudoReturnFields, RegexBytesRefFilterTest, TestCSVLoader, HdfsRestartWhileUpdatingTest, FileBasedSpellCheckerTest, ChangedSchemaMergeTest, TestComplexPhraseQParserPlugin, TestTrie, HdfsBasicDistributedZk2Test, DocValuesMultiTest, TestDistributedMissingSort, TestFieldResource, HdfsUnloadDistributedZkTest, TestGroupingSearch, TestReload, MoveReplicaHDFSFailoverTest, MetricsHandlerTest, TestRestManager, ReplicationFactorTest, ShufflingReplicaListTransformerTest, HdfsDirectoryFactoryTest, HdfsCollectionsAPIDistributedZkTest, TestQueryUtils, TestEmbeddedSolrServerConstructors, TestSort, VMParamsZkACLAndCredentialsProvidersTest, DirectSolrSpellCheckerTest, TestFieldCacheSort, DistributedFacetPivotSmallTest, TestZkChroot, SolrGraphiteReporterTest, TestImplicitCoreProperties, TestSolrJ, AddBlockUpdateTest, DocumentBuilderTest, TestHdfsBackupRestoreCore, TestAnalyzedSuggestions, ConnectionManagerTest, SchemaVersionSpecificBehaviorTest, DebugComponentTest, TestRemoteStreaming, RulesTest, TestPayloadScoreQParserPlugin, TestClusterStateProvider, TestPolicyCloud, TestSearchPerf, TestPolicyCloud, CoreAdminCreateDiscoverTest, ZkStateReaderTest, SolrMetricsIntegrationTest, TestComponentsName, GraphQueryTest, ClassificationUpdateProcessorTest, MigrateRouteKeyTest, TestSolrIndexConfig, TestElisionMultitermQuery, SimpleFacetsTest, TestHttpShardHandlerFactory, TestFastWriter, TestGraphTermsQParserPlugin, PluginInfoTest, TestManagedSchema, PreAnalyzedFieldTest, LeaderFailureAfterFreshStartTest, SparseHLLTest, StatelessScriptUpdateProcessorFactoryTest, SynonymTokenizerTest, TestConfigSetsAPI, TestExtendedDismaxParser, TestConfigReload, StatsReloadRaceTest, AliasIntegrationTest, TestCharFilters, CoreMergeIndexesAdminHandlerTest, BufferStoreTest, LoggingHandlerTest, TestDownShardTolerantSearch, SoftAutoCommitTest, HttpTriggerListenerTest, TestReqParamsAPI, AutoAddReplicasIntegrationTest, TestBM25SimilarityFactory, FileUtilsTest, SortSpecParsingTest, DistributedFacetExistsSmallTest, TestExactStatsCache, TestCopyFieldCollectionResource, SecurityConfHandlerTest, TestSuggestSpellingConverter, SolrJmxReporterTest, SpellingQueryConverterTest, TestReplicaProperties, TestPrepRecovery, TestSolrConfigHandlerCloud, SolrCloudReportersTest, QueryElevationComponentTest, TestNonDefinedSimilarityFactory, DeleteLastCustomShardedReplicaTest, DistributedQueueTest, TestSolrCoreSnapshots, TestLegacyFieldCache, TestCollectionAPIs, ImplicitSnitchTest, TestSystemIdResolver, NodeAddedTriggerTest, TestSchemaVersionResource, TestDocSet, ConcurrentDeleteAndCreateCollectionTest, LeaderElectionContextKeyTest, DistributedSuggestComponentTest, TestDistributedGrouping, TestRandomCollapseQParserPlugin, IndexSchemaRuntimeFieldTest, ParsingFieldUpdateProcessorsTest, TestCollationField, TestSchemaManager, TestConfigSets, SolrIndexConfigTest, SampleTest, TestCustomDocTransformer, CloudMLTQParserTest, DeleteStatusTest, TestCoreContainer, TestGenericDistributedQueue, TestInPlaceUpdatesStandalone, HealthCheckHandlerTest, SpellCheckCollatorWithCollapseTest, BlockJoinFacetRandomTest, BigEndianAscendingWordSerializerTest, DocValuesTest, TestUniqueKeyFieldResource, TestInitQParser, TestCollationFieldDocValues, ConfigureRecoveryStrategyTest, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest, SyncSliceTest, OverseerTest, ShardRoutingTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, ClusterStateUpdateTest, TestRandomFaceting, ZkSolrClientTest, TestRandomDVFaceting, ZkCLITest, ShardRoutingCustomTest, TestDistributedSearch, TestRecovery, TestHashPartitioner, TermVectorComponentDistributedTest, TestStressReorder, TestJoin, TestStressVersions, DistributedTermsComponentTest, TestSolr4Spatial, SolrCoreTest, StatsComponentTest, SolrCmdDistributorTest, BadIndexSchemaTest, ConvertedLegacyTest, TestFiltering, TestFunctionQuery, BasicFunctionalityTest, TestLazyCores, TestBadConfig, TestIndexSearcher, HighlighterTest, SolrIndexSplitterTest, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, SpellCheckCollatorTest, CoreAdminHandlerTest, SolrRequestParserTest, TestFoldingMultitermQuery, SuggesterWFSTTest, PolyFieldTest, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, FieldMutatingUpdateProcessorTest, TestWordDelimiterFilterFactory, QueryEqualityTest, DefaultValueUpdateProcessorTest, DistanceFunctionTest, TestSolrDeletionPolicy1, CacheHeaderTest, IndexBasedSpellCheckerTest, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, TestQueryTypes, TestOmitPositions, TermsComponentTest, XmlUpdateRequestHandlerTest, RequestHandlersTest, TestValueSourceCache, PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, TestIndexingPerformance, MoreLikeThisHandlerTest, FieldAnalysisRequestHandlerTest, TestSolrQueryParser, RegexBoostProcessorTest, SolrPluginUtilsTest, IndexSchemaTest, TestJmxIntegration, UpdateRequestProcessorFactoryTest, MBeansHandlerTest, JsonLoaderTest, BinaryUpdateRequestHandlerTest, TestPHPSerializedResponseWriter, PingRequestHandlerTest, CSVRequestHandlerTest, SearchHandlerTest, HighlighterConfigTest, SOLR749Test, TestQuerySenderListener, TestQuerySenderNoQuery, CopyFieldTest, BadComponentTest, TestStressRecovery, TestMergePolicyConfig, MultiTermTest, NumericFieldsTest, MinimalSchemaTest, ExternalFileFieldSortTest, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestPhraseSuggestions, SpellPossibilityIteratorTest, TestCodecSupport, TestXIncludeConfig, EchoParamsTest, TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestLMJelinekMercerSimilarityFactory, ResourceLoaderTest, TestFastOutputStream, OpenExchangeRatesOrgProviderTest, URLClassifyProcessorTest, TestFastLRUCache, PrimUtilsTest, DOMUtilTest, RAMDirectoryFactoryTest, TestUtils, TestDocumentBuilder, ZkNodePropsTest, SliceStateTest, UUIDFieldTest, TestRTGBase, CursorPagingTest, TestSimpleTrackingShardHandler, TestLuceneIndexBackCompat, TestEmbeddedSolrServerAdminHandler, TestJettySolrRunner, ConnectionReuseTest, ActionThrottleTest, AddReplicaTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, ChaosMonkeySafeLeaderWithPullReplicasTest, CleanupOldIndexTest, CloudExitableDirectoryReaderTest, CollectionReloadTest, CollectionStateFormat2Test, CustomCollectionTest, DeleteInactiveReplicaTest, DeleteReplicaTest, DeleteShardTest, DistribDocExpirationUpdateProcessorTest, DistribJoinFromCollectionTest, HttpPartitionTest, LeaderInitiatedRecoveryOnShardRestartTest, MissingSegmentRecoveryTest, MoveReplicaTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerModifyCollectionTest, OverseerRolesTest, OverseerStatusTest, OverseerTaskQueueTest, PeerSyncReplicationTest, RecoveryAfterSoftCommitTest, RemoteQueryErrorTest, RestartWhileUpdatingTest, RollingRestartTest, SimpleCollectionCreateDeleteTest, SolrCLIZkUtilsTest, SolrCloudExampleTest, SolrXmlInZkTest, TestCloudDeleteByQuery, TestCloudJSONFacetJoinDomain, TestCloudPivotFacet, TestClusterProperties, TestCollectionAPI, TestCollectionsAPIViaSolrCloudCluster, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestDeleteCollectionOnDownNodes, TestExclusionRuleCollectionAccess, TestLeaderElectionWithEmptyReplica, TestLeaderElectionZkExpiry, TestLeaderInitiatedRecoveryThread, TestLocalFSCloudBackupRestore, TestMiniSolrCloudClusterSSL, TestRebalanceLeaders, TestRequestForwarding, TestRequestStatusCollectionAPI, TestSizeLimitedDistributedMap, TestSolrCloudWithSecureImpersonation]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSolrCloudWithSecureImpersonation -Dtests.seed=6C9B73DD0AE286F9 -Dtests.slow=true -Dtests.locale=es-DO -Dtests.timezone=America/Indiana/Indianapolis -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | TestSolrCloudWithSecureImpersonation (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
   [junit4]    >    1) Thread[id=37096, name=jetty-launcher-10176-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
   [junit4]    >         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
   [junit4]    >         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
   [junit4]    >         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
   [junit4]    >         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([6C9B73DD0AE286F9]:0)
   [junit4] Completed [575/766 (1!)] on J1 in 47.68s, 12 tests, 1 error <<< FAILURES!

[...truncated 48701 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1612 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1612/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.handler.TestSolrConfigHandlerConcurrent.test

Error Message:


Stack Trace:
java.lang.NullPointerException
        at __randomizedtesting.SeedInfo.seed([A4068AFD34FC1017:2C52B5279A007DEF]:0)
        at org.apache.solr.handler.TestSolrConfigHandlerConcurrent.test(TestSolrConfigHandlerConcurrent.java:91)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 11976 lines...]
   [junit4] Suite: org.apache.solr.handler.TestSolrConfigHandlerConcurrent
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/init-core-data-001
   [junit4]   2> 649771 WARN  (SUITE-TestSolrConfigHandlerConcurrent-seed#[A4068AFD34FC1017]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=39 numCloses=39
   [junit4]   2> 649772 INFO  (SUITE-TestSolrConfigHandlerConcurrent-seed#[A4068AFD34FC1017]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 649774 INFO  (SUITE-TestSolrConfigHandlerConcurrent-seed#[A4068AFD34FC1017]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 649774 INFO  (SUITE-TestSolrConfigHandlerConcurrent-seed#[A4068AFD34FC1017]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 649777 INFO  (SUITE-TestSolrConfigHandlerConcurrent-seed#[A4068AFD34FC1017]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /c/hl
   [junit4]   2> 649781 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 649781 INFO  (Thread-2521) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 649781 INFO  (Thread-2521) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 649783 ERROR (Thread-2521) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 649881 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.ZkTestServer start zk server on port:34727
   [junit4]   2> 649885 INFO  (zkConnectionManagerCallback-649-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 649892 INFO  (zkConnectionManagerCallback-651-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 649905 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 649910 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 649913 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 649916 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 649919 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 649922 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 649923 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 649925 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 649928 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 649930 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 649936 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 649940 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 650149 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 650151 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@e86cd35{/c/hl,null,AVAILABLE}
   [junit4]   2> 650154 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@28d34f62{HTTP/1.1,[http/1.1]}{127.0.0.1:39110}
   [junit4]   2> 650154 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.Server Started @652384ms
   [junit4]   2> 650154 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/tempDir-001/control/data, replicaType=NRT, hostContext=/c/hl, hostPort=39110, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/control-001/cores}
   [junit4]   2> 650158 ERROR (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 650158 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 650158 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 650158 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 650158 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-07T20:32:47.367Z
   [junit4]   2> 650165 INFO  (zkConnectionManagerCallback-653-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 650168 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 650168 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/control-001/solr.xml
   [junit4]   2> 650174 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 650174 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 650176 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 650180 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34727/solr
   [junit4]   2> 650187 INFO  (zkConnectionManagerCallback-657-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 650196 INFO  (zkConnectionManagerCallback-659-thread-1-processing-n:127.0.0.1:39110_c%2Fhl) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 650509 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 650510 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39110_c%2Fhl
   [junit4]   2> 650516 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.Overseer Overseer (id=99310447295922180-127.0.0.1:39110_c%2Fhl-n_0000000000) starting
   [junit4]   2> 650557 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39110_c%2Fhl
   [junit4]   2> 650592 INFO  (zkCallback-658-thread-1-processing-n:127.0.0.1:39110_c%2Fhl) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 650752 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d
   [junit4]   2> 650763 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d
   [junit4]   2> 650763 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d
   [junit4]   2> 650765 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/control-001/cores
   [junit4]   2> 650800 INFO  (zkConnectionManagerCallback-665-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 650802 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 650803 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34727/solr ready
   [junit4]   2> 650819 INFO  (qtp1762150723-5693) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:39110_c%252Fhl&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 650822 INFO  (OverseerThreadFactory-1335-thread-1-processing-n:127.0.0.1:39110_c%2Fhl) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 650941 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 650942 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 651048 INFO  (zkCallback-658-thread-1-processing-n:127.0.0.1:39110_c%2Fhl) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 651981 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 652041 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 652164 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 652198 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 652198 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d
   [junit4]   2> 652199 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 652199 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 652203 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=44, maxMergeAtOnceExplicit=34, maxMergedSegmentMB=21.67578125, floorSegmentMB=2.076171875, forceMergeDeletesPctAllowed=23.975937833273075, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3664449139759117
   [junit4]   2> 652206 WARN  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 652277 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 652277 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 652278 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 652278 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 652279 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=12.728515625, floorSegmentMB=1.150390625, forceMergeDeletesPctAllowed=25.24987261255103, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.19660181004389932
   [junit4]   2> 652279 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7e589d94[control_collection_shard1_replica_n1] main]
   [junit4]   2> 652281 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 652281 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 652282 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 652282 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588967159356194816
   [junit4]   2> 652283 INFO  (searcherExecutor-1338-thread-1-processing-n:127.0.0.1:39110_c%2Fhl x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@7e589d94[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 652289 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 652289 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 652289 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39110/c/hl/control_collection_shard1_replica_n1/
   [junit4]   2> 652289 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 652289 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:39110/c/hl/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 652289 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 652295 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39110/c/hl/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 652411 INFO  (zkCallback-658-thread-1-processing-n:127.0.0.1:39110_c%2Fhl) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 652447 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 652451 INFO  (qtp1762150723-5694) [n:127.0.0.1:39110_c%2Fhl c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1510
   [junit4]   2> 652460 INFO  (qtp1762150723-5693) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 652558 INFO  (zkCallback-658-thread-1-processing-n:127.0.0.1:39110_c%2Fhl) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 652826 INFO  (OverseerCollectionConfigSetProcessor-99310447295922180-127.0.0.1:39110_c%2Fhl-n_0000000000) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 653489 INFO  (qtp1762150723-5693) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:39110_c%252Fhl&wt=javabin&version=2} status=0 QTime=2670
   [junit4]   2> 653500 INFO  (zkConnectionManagerCallback-669-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 653513 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 653514 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34727/solr ready
   [junit4]   2> 653514 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 653515 INFO  (qtp1762150723-5691) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 653520 INFO  (OverseerThreadFactory-1335-thread-2-processing-n:127.0.0.1:39110_c%2Fhl) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 653527 WARN  (OverseerThreadFactory-1335-thread-2-processing-n:127.0.0.1:39110_c%2Fhl) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 653778 INFO  (qtp1762150723-5691) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 653796 INFO  (qtp1762150723-5691) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=280
   [junit4]   2> 654186 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-1-001 of type NRT
   [junit4]   2> 654209 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 654211 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7604d225{/c/hl,null,AVAILABLE}
   [junit4]   2> 654212 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3d2e9f8d{HTTP/1.1,[http/1.1]}{127.0.0.1:51713}
   [junit4]   2> 654212 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.Server Started @656442ms
   [junit4]   2> 654212 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/c/hl, hostPort=51713, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-1-001/cores}
   [junit4]   2> 654212 ERROR (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 654212 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 654212 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 654213 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 654213 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-07T20:32:51.422Z
   [junit4]   2> 654219 INFO  (zkConnectionManagerCallback-671-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 654221 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 654221 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-1-001/solr.xml
   [junit4]   2> 654227 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 654227 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 654227 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 654233 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34727/solr
   [junit4]   2> 654239 INFO  (zkConnectionManagerCallback-675-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 654270 INFO  (zkConnectionManagerCallback-677-thread-1-processing-n:127.0.0.1:51713_c%2Fhl) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 654284 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 654288 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 654295 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51713_c%2Fhl
   [junit4]   2> 654299 INFO  (zkCallback-668-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 654301 INFO  (zkCallback-658-thread-2-processing-n:127.0.0.1:39110_c%2Fhl) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 654304 INFO  (zkCallback-676-thread-1-processing-n:127.0.0.1:51713_c%2Fhl) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 654587 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d
   [junit4]   2> 654598 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d
   [junit4]   2> 654598 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d
   [junit4]   2> 654602 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-1-001/cores
   [junit4]   2> 654660 INFO  (qtp1762150723-5690) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:51713_c%252Fhl&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 654682 INFO  (OverseerCollectionConfigSetProcessor-99310447295922180-127.0.0.1:39110_c%2Fhl-n_0000000000) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 654684 INFO  (OverseerThreadFactory-1335-thread-3-processing-n:127.0.0.1:39110_c%2Fhl) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:51713_c%2Fhl for creating new replica
   [junit4]   2> 654690 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 654691 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 654835 INFO  (zkCallback-676-thread-1-processing-n:127.0.0.1:51713_c%2Fhl) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 655740 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 655749 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema [collection1_shard2_replica_n41] Schema name=test
   [junit4]   2> 655825 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 655873 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n41' using configuration from collection collection1, trusted=true
   [junit4]   2> 655873 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n41' (registry 'solr.core.collection1.shard2.replica_n41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d
   [junit4]   2> 655873 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 655873 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [[collection1_shard2_replica_n41] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-1-001/cores/collection1_shard2_replica_n41], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-1-001/cores/collection1_shard2_replica_n41/data/]
   [junit4]   2> 655874 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=44, maxMergeAtOnceExplicit=34, maxMergedSegmentMB=21.67578125, floorSegmentMB=2.076171875, forceMergeDeletesPctAllowed=23.975937833273075, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3664449139759117
   [junit4]   2> 655897 WARN  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 655958 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 655958 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 655963 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 655963 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 655964 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=12.728515625, floorSegmentMB=1.150390625, forceMergeDeletesPctAllowed=25.24987261255103, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.19660181004389932
   [junit4]   2> 655964 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.SolrIndexSearcher Opening [Searcher@6f8c4802[collection1_shard2_replica_n41] main]
   [junit4]   2> 655966 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 655967 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 655967 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 655968 INFO  (searcherExecutor-1349-thread-1-processing-n:127.0.0.1:51713_c%2Fhl x:collection1_shard2_replica_n41 s:shard2 c:collection1) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [collection1_shard2_replica_n41] Registered new searcher Searcher@6f8c4802[collection1_shard2_replica_n41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 655969 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588967163222294528
   [junit4]   2> 655975 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 655975 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 655975 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51713/c/hl/collection1_shard2_replica_n41/
   [junit4]   2> 655975 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 655975 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy http://127.0.0.1:51713/c/hl/collection1_shard2_replica_n41/ has no replicas
   [junit4]   2> 655975 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 655981 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51713/c/hl/collection1_shard2_replica_n41/ shard2
   [junit4]   2> 656084 INFO  (zkCallback-676-thread-1-processing-n:127.0.0.1:51713_c%2Fhl) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 656133 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 656159 INFO  (qtp780391273-5737) [n:127.0.0.1:51713_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1469
   [junit4]   2> 656163 INFO  (qtp1762150723-5690) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:51713_c%252Fhl&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1502
   [junit4]   2> 656267 INFO  (zkCallback-676-thread-1-processing-n:127.0.0.1:51713_c%2Fhl) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 656362 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-2-001 of type NRT
   [junit4]   2> 656362 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 656364 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4f85bafb{/c/hl,null,AVAILABLE}
   [junit4]   2> 656365 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2af2fcd1{HTTP/1.1,[http/1.1]}{127.0.0.1:37037}
   [junit4]   2> 656365 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.Server Started @658596ms
   [junit4]   2> 656365 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/c/hl, hostPort=37037, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-2-001/cores}
   [junit4]   2> 656365 ERROR (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 656367 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 656367 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 656367 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 656367 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-07T20:32:53.576Z
   [junit4]   2> 656375 INFO  (zkConnectionManagerCallback-681-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 656378 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 656378 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-2-001/solr.xml
   [junit4]   2> 656383 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 656383 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 656383 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 656392 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34727/solr
   [junit4]   2> 656399 INFO  (zkConnectionManagerCallback-685-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 656414 INFO  (zkConnectionManagerCallback-687-thread-1-processing-n:127.0.0.1:37037_c%2Fhl) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 656426 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 656429 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 656437 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37037_c%2Fhl
   [junit4]   2> 656438 INFO  (zkCallback-668-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 656438 INFO  (zkCallback-658-thread-2-processing-n:127.0.0.1:39110_c%2Fhl) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 656439 INFO  (zkCallback-676-thread-1-processing-n:127.0.0.1:51713_c%2Fhl) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 656443 INFO  (zkCallback-686-thread-1-processing-n:127.0.0.1:37037_c%2Fhl) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 656685 INFO  (OverseerCollectionConfigSetProcessor-99310447295922180-127.0.0.1:39110_c%2Fhl-n_0000000000) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 656774 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d
   [junit4]   2> 656805 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d
   [junit4]   2> 656805 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d
   [junit4]   2> 656809 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-2-001/cores
   [junit4]   2> 656891 INFO  (qtp611203662-5769) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:37037_c%252Fhl&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 656902 INFO  (OverseerThreadFactory-1335-thread-4-processing-n:127.0.0.1:39110_c%2Fhl) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:37037_c%2Fhl for creating new replica
   [junit4]   2> 656915 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 656915 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 657026 INFO  (zkCallback-676-thread-1-processing-n:127.0.0.1:51713_c%2Fhl) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 657026 INFO  (zkCallback-686-thread-1-processing-n:127.0.0.1:37037_c%2Fhl) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 657941 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 657960 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema [collection1_shard1_replica_n43] Schema name=test
   [junit4]   2> 658091 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 658124 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n43' using configuration from collection collection1, trusted=true
   [junit4]   2> 658128 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n43' (registry 'solr.core.collection1.shard1.replica_n43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d
   [junit4]   2> 658128 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 658129 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [[collection1_shard1_replica_n43] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-2-001/cores/collection1_shard1_replica_n43], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-2-001/cores/collection1_shard1_replica_n43/data/]
   [junit4]   2> 658132 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=44, maxMergeAtOnceExplicit=34, maxMergedSegmentMB=21.67578125, floorSegmentMB=2.076171875, forceMergeDeletesPctAllowed=23.975937833273075, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3664449139759117
   [junit4]   2> 658135 WARN  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 658181 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 658181 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 658182 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 658182 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 658197 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=12.728515625, floorSegmentMB=1.150390625, forceMergeDeletesPctAllowed=25.24987261255103, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.19660181004389932
   [junit4]   2> 658198 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.SolrIndexSearcher Opening [Searcher@660805a0[collection1_shard1_replica_n43] main]
   [junit4]   2> 658202 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 658203 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 658203 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 658204 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588967165565861888
   [junit4]   2> 658211 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 658211 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 658211 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37037/c/hl/collection1_shard1_replica_n43/
   [junit4]   2> 658211 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 658211 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy http://127.0.0.1:37037/c/hl/collection1_shard1_replica_n43/ has no replicas
   [junit4]   2> 658211 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 658218 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37037/c/hl/collection1_shard1_replica_n43/ shard1
   [junit4]   2> 658221 INFO  (searcherExecutor-1360-thread-1-processing-n:127.0.0.1:37037_c%2Fhl x:collection1_shard1_replica_n43 s:shard1 c:collection1) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [collection1_shard1_replica_n43] Registered new searcher Searcher@660805a0[collection1_shard1_replica_n43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 658322 INFO  (zkCallback-676-thread-1-processing-n:127.0.0.1:51713_c%2Fhl) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 658327 INFO  (zkCallback-686-thread-1-processing-n:127.0.0.1:37037_c%2Fhl) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 658370 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 658377 INFO  (qtp611203662-5773) [n:127.0.0.1:37037_c%2Fhl c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1462
   [junit4]   2> 658381 INFO  (qtp611203662-5769) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:37037_c%252Fhl&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1491
   [junit4]   2> 658480 INFO  (zkCallback-676-thread-1-processing-n:127.0.0.1:51713_c%2Fhl) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 658480 INFO  (zkCallback-686-thread-1-processing-n:127.0.0.1:37037_c%2Fhl) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 658566 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-3-001 of type NRT
   [junit4]   2> 658567 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 658569 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@63123212{/c/hl,null,AVAILABLE}
   [junit4]   2> 658569 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1d27a096{HTTP/1.1,[http/1.1]}{127.0.0.1:63024}
   [junit4]   2> 658569 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.Server Started @660801ms
   [junit4]   2> 658569 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/c/hl, hostPort=63024, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-3-001/cores}
   [junit4]   2> 658570 ERROR (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 658571 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 658571 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 658571 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 658571 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-07T20:32:55.780Z
   [junit4]   2> 658576 INFO  (zkConnectionManagerCallback-691-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 658580 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 658580 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-3-001/solr.xml
   [junit4]   2> 658583 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 658583 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 658584 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 658587 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34727/solr
   [junit4]   2> 658590 INFO  (zkConnectionManagerCallback-695-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 658597 INFO  (zkConnectionManagerCallback-697-thread-1-processing-n:127.0.0.1:63024_c%2Fhl) [n:127.0.0.1:63024_c%2Fhl    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 658610 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:63024_c%2Fhl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 658613 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:63024_c%2Fhl    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 658617 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:63024_c%2Fhl    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63024_c%2Fhl
   [junit4]   2> 658623 INFO  (zkCallback-658-thread-2-processing-n:127.0.0.1:39110_c%2Fhl) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 658625 INFO  (zkCallback-668-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 658625 INFO  (zkCallback-676-thread-1-processing-n:127.0.0.1:51713_c%2Fhl) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 658625 INFO  (zkCallback-686-thread-1-processing-n:127.0.0.1:37037_c%2Fhl) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 658626 INFO  (zkCallback-696-thread-1-processing-n:127.0.0.1:63024_c%2Fhl) [n:127.0.0.1:63024_c%2Fhl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 658888 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:63024_c%2Fhl    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d
   [junit4]   2> 658900 INFO  (OverseerCollectionConfigSetProcessor-99310447295922180-127.0.0.1:39110_c%2Fhl-n_0000000000) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 658902 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:63024_c%2Fhl    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d
   [junit4]   2> 658902 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:63024_c%2Fhl    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d
   [junit4]   2> 658904 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [n:127.0.0.1:63024_c%2Fhl    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-3-001/cores
   [junit4]   2> 658941 INFO  (qtp611203662-5774) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:63024_c%252Fhl&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 658949 INFO  (OverseerThreadFactory-1335-thread-5-processing-n:127.0.0.1:39110_c%2Fhl) [n:127.0.0.1:39110_c%2Fhl    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:63024_c%2Fhl for creating new replica
   [junit4]   2> 658960 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 658960 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 659068 INFO  (zkCallback-696-thread-1-processing-n:127.0.0.1:63024_c%2Fhl) [n:127.0.0.1:63024_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 659068 INFO  (zkCallback-676-thread-1-processing-n:127.0.0.1:51713_c%2Fhl) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 659068 INFO  (zkCallback-686-thread-1-processing-n:127.0.0.1:37037_c%2Fhl) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 659983 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 659992 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.IndexSchema [collection1_shard2_replica_n45] Schema name=test
   [junit4]   2> 660086 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 660136 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n45' using configuration from collection collection1, trusted=true
   [junit4]   2> 660136 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n45' (registry 'solr.core.collection1.shard2.replica_n45') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dbbe6d
   [junit4]   2> 660136 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 660136 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrCore [[collection1_shard2_replica_n45] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-3-001/cores/collection1_shard2_replica_n45], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-3-001/cores/collection1_shard2_replica_n45/data/]
   [junit4]   2> 660140 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=44, maxMergeAtOnceExplicit=34, maxMergedSegmentMB=21.67578125, floorSegmentMB=2.076171875, forceMergeDeletesPctAllowed=23.975937833273075, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3664449139759117
   [junit4]   2> 660144 WARN  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 660212 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 660212 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 660213 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 660213 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 660215 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=12.728515625, floorSegmentMB=1.150390625, forceMergeDeletesPctAllowed=25.24987261255103, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.19660181004389932
   [junit4]   2> 660215 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.SolrIndexSearcher Opening [Searcher@534f1f12[collection1_shard2_replica_n45] main]
   [junit4]   2> 660218 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 660218 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 660219 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 660219 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588967167678742528
   [junit4]   2> 660220 INFO  (searcherExecutor-1371-thread-1-processing-n:127.0.0.1:63024_c%2Fhl x:collection1_shard2_replica_n45 s:shard2 c:collection1) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.SolrCore [collection1_shard2_replica_n45] Registered new searcher Searcher@534f1f12[collection1_shard2_replica_n45] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 660226 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.c.ZkController Core needs to recover:collection1_shard2_replica_n45
   [junit4]   2> 660227 INFO  (updateExecutor-692-thread-1-processing-n:127.0.0.1:63024_c%2Fhl x:collection1_shard2_replica_n45 s:shard2 c:collection1) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 660228 INFO  (recoveryExecutor-693-thread-1-processing-n:127.0.0.1:63024_c%2Fhl x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 660228 INFO  (recoveryExecutor-693-thread-1-processing-n:127.0.0.1:63024_c%2Fhl x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 660228 INFO  (recoveryExecutor-693-thread-1-processing-n:127.0.0.1:63024_c%2Fhl x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard2_replica_n45]
   [junit4]   2> 660228 INFO  (recoveryExecutor-693-thread-1-processing-n:127.0.0.1:63024_c%2Fhl x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 660228 INFO  (recoveryExecutor-693-thread-1-processing-n:127.0.0.1:63024_c%2Fhl x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard2_replica_n45] as recovering, leader is [http://127.0.0.1:51713/c/hl/collection1_shard2_replica_n41/] and I am [http://127.0.0.1:63024/c/hl/collection1_shard2_replica_n45/]
   [junit4]   2> 660228 INFO  (qtp1449366379-5800) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2  x:collection1_shard2_replica_n45] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1268
   [junit4]   2> 660234 INFO  (qtp611203662-5774) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:63024_c%252Fhl&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1293
   [junit4]   2> 660239 INFO  (recoveryExecutor-693-thread-1-processing-n:127.0.0.1:63024_c%2Fhl x:collection1_shard2_replica_n45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:63024_c%2Fhl c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_n45] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:51713/c/hl]; [WaitForState: action=PREPRECOVERY&core=collection1_shard2_replica_n41&nodeName=127.0.0.1:63024_c%252Fhl&coreNodeName=core_node46&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 660241 INFO  (qtp780391273-5739) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node46, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
   [junit4]   2> 660243 INFO  (qtp780391273-5739) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1_shard2_replica_n41, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:63024_c%2Fhl, coreNodeName=core_node46, onlyIfActiveCheckResult=false, nodeProps: core_node46:{"core":"collection1_shard2_replica_n45","base_url":"http://127.0.0.1:63024/c/hl","node_name":"127.0.0.1:63024_c%2Fhl","state":"down","type":"NRT"}
   [junit4]   2> 660342 INFO  (zkCallback-696-thread-1-processing-n:127.0.0.1:63024_c%2Fhl) [n:127.0.0.1:63024_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 660343 INFO  (zkCallback-686-thread-1-processing-n:127.0.0.1:37037_c%2Fhl) [n:127.0.0.1:37037_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 660344 INFO  (zkCallback-676-thread-1-processing-n:127.0.0.1:51713_c%2Fhl) [n:127.0.0.1:51713_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 660412 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/shard-4-001 of type NRT
   [junit4]   2> 660413 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 660414 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5b38fdf6{/c/hl,null,AVAILABLE}
   [junit4]   2> 660415 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@267acedd{HTTP/1.1,[http/1.1]}{127.0.0.1:39453}
   [junit4]   2> 660415 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.Server Started @662647ms
   [junit4]   2> 660415 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001/tempDir-001/jetty4, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/c/hl, hostPort=39453, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrC

[...truncated too long message...]

ContextHandler Stopped o.e.j.s.ServletContextHandler@63123212{/c/hl,null,UNAVAILABLE}
   [junit4]   2> 667003 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.ChaosMonkey monkey: stop jetty! 39453
   [junit4]   2> 667003 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1963578608
   [junit4]   2> 667004 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 667004 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4ad4f459: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@769b5d8c
   [junit4]   2> 667013 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 667013 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@d7fe866: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5f747b48
   [junit4]   2> 667019 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 667020 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4b050d71: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@3bf61bb9
   [junit4]   2> 667029 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:39453_c%2Fhl
   [junit4]   2> 667032 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 667037 INFO  (zkCallback-707-thread-2-processing-n:127.0.0.1:39453_c%2Fhl) [n:127.0.0.1:39453_c%2Fhl    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 667041 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.Overseer Overseer (id=99310447295922194-127.0.0.1:39453_c%2Fhl-n_0000000004) closing
   [junit4]   2> 667047 INFO  (OverseerStateUpdate-99310447295922194-127.0.0.1:39453_c%2Fhl-n_0000000004) [n:127.0.0.1:39453_c%2Fhl    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:39453_c%2Fhl
   [junit4]   2> 667063 INFO  (zkCallback-707-thread-3-processing-n:127.0.0.1:39453_c%2Fhl) [n:127.0.0.1:39453_c%2Fhl    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [0])
   [junit4]   2> 667063 WARN  (OverseerAutoScalingTriggerThread-99310447295922194-127.0.0.1:39453_c%2Fhl-n_0000000004) [n:127.0.0.1:39453_c%2Fhl    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 667095 WARN  (zkCallback-707-thread-1-processing-n:127.0.0.1:39453_c%2Fhl) [n:127.0.0.1:39453_c%2Fhl c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_n47] o.a.s.c.SyncStrategy Closed, skipping sync up.
   [junit4]   2> 667095 INFO  (zkCallback-707-thread-1-processing-n:127.0.0.1:39453_c%2Fhl) [n:127.0.0.1:39453_c%2Fhl c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_n47] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 667095 INFO  (zkCallback-707-thread-1-processing-n:127.0.0.1:39453_c%2Fhl) [n:127.0.0.1:39453_c%2Fhl c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_n47] o.a.s.c.SolrCore [collection1_shard1_replica_n47]  CLOSING SolrCore org.apache.solr.core.SolrCore@5e9fc785
   [junit4]   2> 667095 INFO  (zkCallback-707-thread-1-processing-n:127.0.0.1:39453_c%2Fhl) [n:127.0.0.1:39453_c%2Fhl c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_n47] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_n47, tag=1587529605
   [junit4]   2> 667095 INFO  (zkCallback-707-thread-1-processing-n:127.0.0.1:39453_c%2Fhl) [n:127.0.0.1:39453_c%2Fhl c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_n47] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2fd58b2f: rootName = null, domain = solr.core.collection1.shard1.replica_n47, service url = null, agent id = null] for registry solr.core.collection1.shard1.replica_n47 / com.codahale.metrics.MetricRegistry@3f37694b
   [junit4]   2> 667110 INFO  (zkCallback-707-thread-1-processing-n:127.0.0.1:39453_c%2Fhl) [n:127.0.0.1:39453_c%2Fhl c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_n47] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=1587529605
   [junit4]   2> 667115 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@267acedd{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 667115 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5b38fdf6{/c/hl,null,UNAVAILABLE}
   [junit4]   2> 667117 ERROR (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 667120 INFO  (TEST-TestSolrConfigHandlerConcurrent.test-seed#[A4068AFD34FC1017]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34727 34727
   [junit4]   2> 667135 INFO  (Thread-2521) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34727 34727
   [junit4]   2> 667136 WARN  (Thread-2521) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 23 /solr/configs/conf1
   [junit4]   2> 7 /solr/aliases.json
   [junit4]   2> 5 /solr/security.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 7 /solr/clusterstate.json
   [junit4]   2> 7 /solr/clusterprops.json
   [junit4]   2> 5 /solr/autoscaling.json
   [junit4]   2> 4 /solr/collections/collection1/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 7 /solr/live_nodes
   [junit4]   2> 7 /solr/collections
   [junit4]   2> 5 /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 5 /solr/overseer/collection-queue-work
   [junit4]   2> 5 /solr/overseer/queue-work
   [junit4]   2> 4 /solr/overseer/queue
   [junit4]   2>
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSolrConfigHandlerConcurrent -Dtests.method=test -Dtests.seed=A4068AFD34FC1017 -Dtests.slow=true -Dtests.locale=lt-LT -Dtests.timezone=Pacific/Chuuk -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   17.4s J0 | TestSolrConfigHandlerConcurrent.test <<<
   [junit4]    > Throwable #1: java.lang.NullPointerException
   [junit4]    > at __randomizedtesting.SeedInfo.seed([A4068AFD34FC1017:2C52B5279A007DEF]:0)
   [junit4]    > at org.apache.solr.handler.TestSolrConfigHandlerConcurrent.test(TestSolrConfigHandlerConcurrent.java:91)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_A4068AFD34FC1017-001
   [junit4]   2> Jan 07, 2018 8:33:04 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1867, maxMBSortInHeap=7.743094638781837, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@2897e197), locale=lt-LT, timezone=Pacific/Chuuk
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=183549480,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [TestMinMaxOnMultiValuedField, TestNumericRangeQuery32, FieldAnalysisRequestHandlerTest, TestPointFields, SubstringBytesRefFilterTest, TestStressCloudBlindAtomicUpdates, ShardSplitTest, AddSchemaFieldsUpdateProcessorFactoryTest, PrimUtilsTest, TestShardHandlerFactory, IndexSchemaTest, TestFieldTypeCollectionResource, ChaosMonkeyNothingIsSafeTest, SolrRequestParserTest, TestFileDictionaryLookup, DisMaxRequestHandlerTest, CurrencyFieldTypeTest, TestCustomSort, TestMissingGroups, MoveReplicaHDFSTest, TestSolrConfigHandlerConcurrent]
   [junit4] Completed [99/766 (1!)] on J0 in 18.20s, 1 test, 1 error <<< FAILURES!

[...truncated 50243 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1613 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1613/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:     1) Thread[id=746, name=jetty-launcher-169-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
   1) Thread[id=746, name=jetty-launcher-169-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
        at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
        at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
        at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
        at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
        at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
        at __randomizedtesting.SeedInfo.seed([93E0442371F0484A]:0)




Build Log:
[...truncated 11679 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
   [junit4]   2> 53810 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[93E0442371F0484A]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_93E0442371F0484A-001/init-core-data-001
   [junit4]   2> 53810 WARN  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[93E0442371F0484A]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=81 numCloses=81
   [junit4]   2> 53810 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[93E0442371F0484A]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 53812 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[93E0442371F0484A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 53861 WARN  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[93E0442371F0484A]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   2> 53975 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[93E0442371F0484A]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_93E0442371F0484A-001/tempDir-001
   [junit4]   2> 53976 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[93E0442371F0484A]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 53976 INFO  (Thread-121) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 53976 INFO  (Thread-121) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 53978 ERROR (Thread-121) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 54076 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[93E0442371F0484A]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:59021
   [junit4]   2> 54079 INFO  (zkConnectionManagerCallback-172-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 54101 INFO  (jetty-launcher-169-thread-1) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 54101 INFO  (jetty-launcher-169-thread-2) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 54104 INFO  (jetty-launcher-169-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5bbdbf9c{/solr,null,AVAILABLE}
   [junit4]   2> 54105 INFO  (jetty-launcher-169-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2bdcc1c2{/solr,null,AVAILABLE}
   [junit4]   2> 54131 INFO  (jetty-launcher-169-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@67e0ff08{SSL,[ssl, http/1.1]}{127.0.0.1:๕๖๒๖๔}
   [junit4]   2> 54131 INFO  (jetty-launcher-169-thread-1) [    ] o.e.j.s.Server Started @๕๕๙๗๗ms
   [junit4]   2> 54131 INFO  (jetty-launcher-169-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=56264}
   [junit4]   2> 54131 ERROR (jetty-launcher-169-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 54131 INFO  (jetty-launcher-169-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 54131 INFO  (jetty-launcher-169-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 54131 INFO  (jetty-launcher-169-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 54131 INFO  (jetty-launcher-169-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-08T07:30:12.025Z
   [junit4]   2> 54135 INFO  (jetty-launcher-169-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@6cc4cb5f{SSL,[ssl, http/1.1]}{127.0.0.1:๓๘๖๑๖}
   [junit4]   2> 54135 INFO  (jetty-launcher-169-thread-2) [    ] o.e.j.s.Server Started @๕๕๙๘๑ms
   [junit4]   2> 54135 INFO  (jetty-launcher-169-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38616}
   [junit4]   2> 54136 ERROR (jetty-launcher-169-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 54136 INFO  (jetty-launcher-169-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 54136 INFO  (jetty-launcher-169-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 54136 INFO  (jetty-launcher-169-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 54136 INFO  (jetty-launcher-169-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-08T07:30:12.030Z
   [junit4]   2> 54139 INFO  (zkConnectionManagerCallback-176-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 54140 INFO  (jetty-launcher-169-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 54142 INFO  (zkConnectionManagerCallback-174-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 54145 INFO  (jetty-launcher-169-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 54159 INFO  (jetty-launcher-169-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59021/solr
   [junit4]   2> 54161 INFO  (jetty-launcher-169-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59021/solr
   [junit4]   2> 54161 INFO  (zkConnectionManagerCallback-180-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 54163 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x160d4ad3f1f0003, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 54164 INFO  (zkConnectionManagerCallback-184-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 54166 INFO  (zkConnectionManagerCallback-186-thread-1-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 54167 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x160d4ad3f1f0004, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 54172 INFO  (zkConnectionManagerCallback-188-thread-1-processing-n:127.0.0.1:38616_solr) [n:127.0.0.1:38616_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 54316 INFO  (jetty-launcher-169-thread-1) [n:127.0.0.1:56264_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 54318 INFO  (jetty-launcher-169-thread-1) [n:127.0.0.1:56264_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56264_solr
   [junit4]   2> 54320 INFO  (jetty-launcher-169-thread-1) [n:127.0.0.1:56264_solr    ] o.a.s.c.Overseer Overseer (id=99313032355577861-127.0.0.1:56264_solr-n_0000000000) starting
   [junit4]   2> 54327 INFO  (jetty-launcher-169-thread-2) [n:127.0.0.1:38616_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 54331 INFO  (jetty-launcher-169-thread-2) [n:127.0.0.1:38616_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38616_solr
   [junit4]   2> 54336 INFO  (zkCallback-187-thread-1-processing-n:127.0.0.1:38616_solr) [n:127.0.0.1:38616_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 54336 INFO  (zkCallback-185-thread-1-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 54348 INFO  (jetty-launcher-169-thread-1) [n:127.0.0.1:56264_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56264_solr
   [junit4]   2> 54352 INFO  (zkCallback-185-thread-1-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 54352 INFO  (zkCallback-187-thread-1-processing-n:127.0.0.1:38616_solr) [n:127.0.0.1:38616_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 54408 INFO  (jetty-launcher-169-thread-1) [n:127.0.0.1:56264_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
   [junit4]   2> 54412 WARN  (jetty-launcher-169-thread-1) [n:127.0.0.1:56264_solr    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 54417 INFO  (jetty-launcher-169-thread-2) [n:127.0.0.1:38616_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
   [junit4]   2> 54417 WARN  (jetty-launcher-169-thread-2) [n:127.0.0.1:38616_solr    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 54424 INFO  (jetty-launcher-169-thread-1) [n:127.0.0.1:56264_solr    ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
   [junit4]   2> 54424 INFO  (jetty-launcher-169-thread-2) [n:127.0.0.1:38616_solr    ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
   [junit4]   2> 54549 INFO  (jetty-launcher-169-thread-2) [n:127.0.0.1:38616_solr    ] o.a.c.f.i.CuratorFrameworkImpl Starting
   [junit4]   2> 54550 INFO  (jetty-launcher-169-thread-1) [n:127.0.0.1:56264_solr    ] o.a.c.f.i.CuratorFrameworkImpl Starting
   [junit4]   2> 54583 INFO  (jetty-launcher-169-thread-1-EventThread) [n:127.0.0.1:56264_solr    ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
   [junit4]   2> 54584 INFO  (jetty-launcher-169-thread-2-EventThread) [n:127.0.0.1:38616_solr    ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
   [junit4]   2> 54757 WARN  (jetty-launcher-169-thread-1) [n:127.0.0.1:56264_solr    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 54758 WARN  (jetty-launcher-169-thread-2) [n:127.0.0.1:38616_solr    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 54909 INFO  (jetty-launcher-169-thread-2) [n:127.0.0.1:38616_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38616.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@31b5519d
   [junit4]   2> 54915 INFO  (jetty-launcher-169-thread-2) [n:127.0.0.1:38616_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38616.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@31b5519d
   [junit4]   2> 54916 INFO  (jetty-launcher-169-thread-2) [n:127.0.0.1:38616_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38616.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@31b5519d
   [junit4]   2> 54917 INFO  (jetty-launcher-169-thread-2) [n:127.0.0.1:38616_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_93E0442371F0484A-001/tempDir-001/node2/.
   [junit4]   2> 54921 INFO  (jetty-launcher-169-thread-1) [n:127.0.0.1:56264_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56264.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@31b5519d
   [junit4]   2> 54926 INFO  (jetty-launcher-169-thread-1) [n:127.0.0.1:56264_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56264.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@31b5519d
   [junit4]   2> 54927 INFO  (jetty-launcher-169-thread-1) [n:127.0.0.1:56264_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56264.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@31b5519d
   [junit4]   2> 54929 INFO  (jetty-launcher-169-thread-1) [n:127.0.0.1:56264_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_93E0442371F0484A-001/tempDir-001/node1/.
   [junit4]   2> 54991 INFO  (zkConnectionManagerCallback-194-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 55008 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateAnyHostAnyUser
   [junit4]   2> 55119 INFO  (qtp171289061-705) [n:127.0.0.1:38616_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 55119 INFO  (qtp171289061-705) [n:127.0.0.1:38616_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 55125 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[93E0442371F0484A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostAnyUser&p=anyHostAnyUser&t=test&e=1515432612995&s=JbcAee+LEs64jME44b35aRJ8Z5Q="; Path=/; Domain=127.0.0.1; Expires= .,   - . .-       :  :   GMT; Secure; HttpOnly". Invalid 'expires' attribute: ., - . .- : : GMT
   [junit4]   2> 55125 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateAnyHostAnyUser
   [junit4]   2> 55142 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidRemote
   [junit4]   2> 55547 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[93E0442371F0484A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1515432613037&s=ZBYL8C4iAYmmwljlY1WsIpk7IxQ="; Path=/; Domain=127.0.0.1; Expires= .,   - . .-       :  :   GMT; Secure; HttpOnly". Invalid 'expires' attribute: ., - . .- : : GMT
   [junit4]   2> 55548 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidRemote
   [junit4]   2> 55563 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidProxyUser
   [junit4]   2> 55570 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[93E0442371F0484A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bar&p=bar&t=test&e=1515432613458&s=m4xjbMzMIkeYqKOVlI354A58XCQ="; Path=/; Domain=127.0.0.1; Expires= .,   - . .-       :  :   GMT; Secure; HttpOnly". Invalid 'expires' attribute: ., - . .- : : GMT
   [junit4]   2> 55570 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidProxyUser
   [junit4]   2> 55587 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testForwarding
   [junit4]   2> 55593 INFO  (zkConnectionManagerCallback-197-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 56167 INFO  (zkConnectionManagerCallback-200-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 56169 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[93E0442371F0484A]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 56171 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[93E0442371F0484A]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59021/solr ready
   [junit4]   2> 56179 INFO  (qtp171289061-705) [n:127.0.0.1:38616_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 56183 INFO  (OverseerThreadFactory-667-thread-1-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr    ] o.a.s.c.CreateCollectionCmd Create collection forwardingCollection
   [junit4]   2> 56308 INFO  (OverseerStateUpdate-99313032355577861-127.0.0.1:56264_solr-n_0000000000) [n:127.0.0.1:56264_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"forwardingCollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"forwardingCollection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:56264/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 56554 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 56555 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 56664 INFO  (zkCallback-185-thread-1-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 56666 INFO  (zkCallback-185-thread-2-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 57594 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 57617 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema [forwardingCollection_shard1_replica_n1] Schema name=test
   [junit4]   2> 57712 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 57741 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'forwardingCollection_shard1_replica_n1' using configuration from collection forwardingCollection, trusted=true
   [junit4]   2> 57742 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56264.solr.core.forwardingCollection.shard1.replica_n1' (registry 'solr.core.forwardingCollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@31b5519d
   [junit4]   2> 57742 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 57742 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [[forwardingCollection_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_93E0442371F0484A-001/tempDir-001/node1/forwardingCollection_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_93E0442371F0484A-001/tempDir-001/node1/./forwardingCollection_shard1_replica_n1/data/]
   [junit4]   2> 57744 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=692240723, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 57798 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 57798 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 57799 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 57799 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 57800 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=24, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 57800 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@520c7795[forwardingCollection_shard1_replica_n1] main]
   [junit4]   2> 57802 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 57807 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 57829 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 57829 INFO  (searcherExecutor-672-thread-1-processing-n:127.0.0.1:56264_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 57829 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1589008521728360448
   [junit4]   2> 57830 INFO  (searcherExecutor-672-thread-1-processing-n:127.0.0.1:56264_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 57830 INFO  (searcherExecutor-672-thread-1-processing-n:127.0.0.1:56264_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 57830 INFO  (searcherExecutor-672-thread-1-processing-n:127.0.0.1:56264_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 57830 INFO  (searcherExecutor-672-thread-1-processing-n:127.0.0.1:56264_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 57830 INFO  (searcherExecutor-672-thread-1-processing-n:127.0.0.1:56264_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 57830 INFO  (searcherExecutor-672-thread-1-processing-n:127.0.0.1:56264_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 57831 INFO  (searcherExecutor-672-thread-1-processing-n:127.0.0.1:56264_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 57831 INFO  (searcherExecutor-672-thread-1-processing-n:127.0.0.1:56264_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 57832 INFO  (searcherExecutor-672-thread-1-processing-n:127.0.0.1:56264_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] Registered new searcher Searcher@520c7795[forwardingCollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 57837 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 57837 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 57837 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:56264/solr/forwardingCollection_shard1_replica_n1/
   [junit4]   2> 57837 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 57837 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:56264/solr/forwardingCollection_shard1_replica_n1/ has no replicas
   [junit4]   2> 57837 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 57843 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:56264/solr/forwardingCollection_shard1_replica_n1/ shard1
   [junit4]   2> 57946 INFO  (zkCallback-185-thread-2-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 57946 INFO  (zkCallback-185-thread-3-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 57995 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 57998 INFO  (qtp2132470970-701) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1444
   [junit4]   2> 57999 WARN  (httpShardExecutor-663-thread-1-processing-n:127.0.0.1:56264_solr [https:////127.0.0.1:56264//solr] https:////127.0.0.1:56264//solr) [n:127.0.0.1:56264_solr    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1515432614448&s=HMo8fj4uxajz0Yhu5YpvgPd3c50="; Path=/; Domain=127.0.0.1; Expires= .,   - . .-       :  :   GMT; Secure; HttpOnly". Invalid 'expires' attribute: ., - . .- : : GMT
   [junit4]   2> 58003 INFO  (qtp171289061-705) [n:127.0.0.1:38616_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 58098 INFO  (zkCallback-185-thread-1-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 58098 INFO  (zkCallback-185-thread-3-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 58185 INFO  (OverseerCollectionConfigSetProcessor-99313032355577861-127.0.0.1:56264_solr-n_0000000000) [n:127.0.0.1:56264_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 59005 INFO  (qtp171289061-705) [n:127.0.0.1:38616_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=2827
   [junit4]   2> 59006 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[93E0442371F0484A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1515432614072&s=MLXytghtQphLMDhfxk8oj1/+LKg="; Path=/; Domain=127.0.0.1; Expires= .,   - . .-       :  :   GMT; Secure; HttpOnly". Invalid 'expires' attribute: ., - . .- : : GMT
   [junit4]   2> 59006 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[93E0442371F0484A]) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: forwardingCollection failOnTimeout:true timeout (sec):100
   [junit4]   2> 59007 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[93E0442371F0484A]) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: forwardingCollection
   [junit4]   2> 59015 INFO  (qtp171289061-703) [n:127.0.0.1:38616_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 59025 INFO  (qtp2132470970-700) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 59026 WARN  (qtp171289061-703) [n:127.0.0.1:38616_solr    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1515432616918&s=wDl7ERFMz7fFj9ZuSaFhi5Ew7Ow="; Path=/; Domain=127.0.0.1; Expires= .,   - . .-       :  :   GMT; Secure; HttpOnly". Invalid 'expires' attribute: ., - . .- : : GMT
   [junit4]   2> 59030 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[93E0442371F0484A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1515432616909&s=BtVCHG0bz7XvsB0j0Rd+ZK2Ze4k="; Path=/; Domain=127.0.0.1; Expires= .,   - . .-       :  :   GMT; Secure; HttpOnly". Invalid 'expires' attribute: ., - . .- : : GMT
   [junit4]   2> 59030 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[93E0442371F0484A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1515432616918&s=wDl7ERFMz7fFj9ZuSaFhi5Ew7Ow="; Path=/; Domain=127.0.0.1; Expires= .,   - . .-       :  :   GMT; Secure; HttpOnly". Invalid 'expires' attribute: ., - . .- : : GMT
   [junit4]   2> 59038 INFO  (qtp2132470970-697) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 59039 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[93E0442371F0484A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1515432616932&s=M22AJFsnQAwWLQNF4SxRmwdhy+A="; Path=/; Domain=127.0.0.1; Expires= .,   - . .-       :  :   GMT; Secure; HttpOnly". Invalid 'expires' attribute: ., - . .- : : GMT
   [junit4]   2> 59039 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testForwarding
   [junit4]   2> 59061 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateGroup
   [junit4]   2> 59142 INFO  (qtp171289061-704) [n:127.0.0.1:38616_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=jenkins&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 59142 INFO  (qtp171289061-704) [n:127.0.0.1:38616_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=jenkins&action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 59143 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[93E0442371F0484A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostUsersGroup&p=anyHostUsersGroup&t=test&e=1515432616957&s=haIzET5hlHx9fI1dzbwEjFL+3Ok="; Path=/; Domain=127.0.0.1; Expires= .,   - . .-       :  :   GMT; Secure; HttpOnly". Invalid 'expires' attribute: ., - . .- : : GMT
   [junit4]   2> 59143 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateGroup
   [junit4]   2> 59164 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidGroup
   [junit4]   2> 59212 WARN  (qtp171289061-705) [n:127.0.0.1:38616_solr    ] o.a.h.s.ShellBasedUnixGroupsMapping got exception trying to get groups for user bar: id: invalid user name: "bar"
   [junit4]   2> id: invalid user name: "bar"
   [junit4]   2>
   [junit4]   2> 59215 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[93E0442371F0484A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bogusGroup&p=bogusGroup&t=test&e=1515432617059&s=XtqRcXE5DWbUDeMXfEb8mN6z9Is="; Path=/; Domain=127.0.0.1; Expires= .,   - . .-       :  :   GMT; Secure; HttpOnly". Invalid 'expires' attribute: ., - . .- : : GMT
   [junit4]   2> 59215 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidGroup
   [junit4]   2> 59244 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNullProxyUser
   [junit4]   2> 59246 INFO  (qtp171289061-708) [    ] o.a.s.s.SolrDispatchFilter Error authenticating
   [junit4]   2> java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
   [junit4]   2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
   [junit4]   2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
   [junit4]   2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
   [junit4]   2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
   [junit4]   2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
   [junit4]   2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
   [junit4]   2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
   [junit4]   2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
   [junit4]   2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
   [junit4]   2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:451)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
   [junit4]   2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
   [junit4]   2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
   [junit4]   2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 59246 WARN  (qtp171289061-708) [    ] o.e.j.s.ServletHandler /solr/admin/collections
   [junit4]   2> org.apache.solr.common.SolrException: Error during request authentication,
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:457)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
   [junit4]   2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
   [junit4]   2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
   [junit4]   2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
   [junit4]   2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
   [junit4]   2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
   [junit4]   2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
   [junit4]   2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
   [junit4]   2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
   [junit4]   2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
   [junit4]   2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
   [junit4]   2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
   [junit4]   2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
   [junit4]   2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:451)
   [junit4]   2> ... 29 more
   [junit4]   2> 59251 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNullProxyUser
   [junit4]   2> 59277 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyUnknownRemote
   [junit4]   2> 69392 WARN  (jetty-launcher-169-thread-1-SendThread(127.0.0.1:59021)) [n:127.0.0.1:56264_solr    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 14646ms for sessionid 0x160d4ad3f1f0007
   [junit4]   2> 69392 WARN  (jetty-launcher-169-thread-2-SendThread(127.0.0.1:59021)) [n:127.0.0.1:38616_solr    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 14645ms for sessionid 0x160d4ad3f1f0008
   [junit4]   2> 69393 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x160d4ad3f1f0008, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 69394 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x160d4ad3f1f0007, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 69400 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[93E0442371F0484A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1515432617178&s=LOXaxWcS8TAmwsnz0T/Jol/5D3E="; Path=/; Domain=127.0.0.1; Expires= .,   - . .-       :  :   GMT; Secure; HttpOnly". Invalid 'expires' attribute: ., - . .- : : GMT
   [junit4]   2> 69400 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyUnknownRemote
   [junit4]   2> 69461 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigHosts
   [junit4]   2> 69471 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[93E0442371F0484A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noHosts&p=noHosts&t=test&e=1515432627358&s=C1p5pCmBKru//Mkh+2Ba/6k1IUc="; Path=/; Domain=127.0.0.1; Expires= .,   - . .-       :  :   GMT; Secure; HttpOnly". Invalid 'expires' attribute: ., - . .- : : GMT
   [junit4]   2> 69472 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigHosts
   [junit4]   2> 69551 WARN  (zkConnectionManagerCallback-200-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7beb574f name: ZooKeeperConnection Watcher:127.0.0.1:59021/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 69551 WARN  (zkConnectionManagerCallback-200-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 69552 INFO  (jetty-launcher-169-thread-2-EventThread) [n:127.0.0.1:38616_solr    ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
   [junit4]   2> 69591 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateHost
   [junit4]   2> 69593 INFO  (qtp171289061-704) [n:127.0.0.1:38616_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 69594 INFO  (qtp171289061-704) [n:127.0.0.1:38616_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 69595 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[93E0442371F0484A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1515432627486&s=GhBcAiCdvfMPRUIbSVP9L8+fmWA="; Path=/; Domain=127.0.0.1; Expires= .,   - . .-       :  :   GMT; Secure; HttpOnly". Invalid 'expires' attribute: ., - . .- : : GMT
   [junit4]   2> 69595 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateHost
   [junit4]   2> 69654 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyWrongHost
   [junit4]   2> 69680 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[93E0442371F0484A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=wrongHost&p=wrongHost&t=test&e=1515432627549&s=DVSZoDEro5O2y4bbYn4McZ5+VNs="; Path=/; Domain=127.0.0.1; Expires= .,   - . .-       :  :   GMT; Secure; HttpOnly". Invalid 'expires' attribute: ., - . .- : : GMT
   [junit4]   2> 69680 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyWrongHost
   [junit4]   2> 69728 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigGroups
   [junit4]   2> 69733 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[93E0442371F0484A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noGroups&p=noGroups&t=test&e=1515432627624&s=/DPAL0HwYlTqRXqlDgxYyTAGq/0="; Path=/; Domain=127.0.0.1; Expires= .,   - . .-       :  :   GMT; Secure; HttpOnly". Invalid 'expires' attribute: ., - . .- : : GMT
   [junit4]   2> 69734 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[93E0442371F0484A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigGroups
   [junit4]   2> 70431 INFO  (jetty-closer-170-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@6cc4cb5f{SSL,[ssl, http/1.1]}{127.0.0.1:๐}
   [junit4]   2> 70431 INFO  (jetty-closer-170-thread-2) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@67e0ff08{SSL,[ssl, http/1.1]}{127.0.0.1:๐}
   [junit4]   2> 70432 INFO  (jetty-closer-170-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=816244732
   [junit4]   2> 70433 INFO  (jetty-closer-170-thread-2) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1900276979
   [junit4]   2> 70433 INFO  (jetty-closer-170-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 70433 INFO  (jetty-closer-170-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6fd5a848: rootName = solr_38616, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@27b3b75b
   [junit4]   2> 70437 INFO  (jetty-closer-170-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 70437 INFO  (jetty-closer-170-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@642667a1: rootName = solr_56264, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@341ef5fa
   [junit4]   2> 70441 INFO  (jetty-closer-170-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 70441 INFO  (jetty-closer-170-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@38a9fba1: rootName = solr_38616, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@1769560f
   [junit4]   2> 70461 INFO  (jetty-closer-170-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 70461 INFO  (jetty-closer-170-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5876fd55: rootName = solr_56264, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@1769560f
   [junit4]   2> 70462 INFO  (jetty-closer-170-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 70462 INFO  (jetty-closer-170-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2eacd15f: rootName = solr_38616, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@60d26ccd
   [junit4]   2> 70464 INFO  (jetty-closer-170-thread-1) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:38616_solr
   [junit4]   2> 70465 INFO  (jetty-closer-170-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 70465 INFO  (jetty-closer-170-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 70466 INFO  (jetty-closer-170-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@cfe63c4: rootName = solr_56264, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@60d26ccd
   [junit4]   2> 70467 INFO  (zkCallback-187-thread-1-processing-n:127.0.0.1:38616_solr) [n:127.0.0.1:38616_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 70467 INFO  (zkCallback-185-thread-1-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 70471 INFO  (jetty-closer-170-thread-2) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:56264_solr
   [junit4]   2> 70474 INFO  (zkCallback-185-thread-1-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 70474 INFO  (zkCallback-185-thread-3-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 70480 INFO  (jetty-closer-170-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 70481 INFO  (coreCloseExecutor-678-thread-1) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@585b8565
   [junit4]   2> 70481 INFO  (coreCloseExecutor-678-thread-1) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.forwardingCollection.shard1.replica_n1, tag=1482392933
   [junit4]   2> 70481 INFO  (coreCloseExecutor-678-thread-1) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@ccf099d: rootName = solr_56264, domain = solr.core.forwardingCollection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.forwardingCollection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@5aec2483
   [junit4]   2> 70482 ERROR (Thread[Thread-138,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:38616_solr    ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 70485 INFO  (zkCallback-185-thread-3-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 70495 INFO  (coreCloseExecutor-678-thread-1) [n:127.0.0.1:56264_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.forwardingCollection.shard1.leader, tag=1482392933
   [junit4]   2> 70501 INFO  (jetty-closer-170-thread-2) [    ] o.a.s.c.Overseer Overseer (id=99313032355577861-127.0.0.1:56264_solr-n_0000000000) closing
   [junit4]   2> 70501 INFO  (OverseerStateUpdate-99313032355577861-127.0.0.1:56264_solr-n_0000000000) [n:127.0.0.1:56264_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:56264_solr
   [junit4]   2> 70505 WARN  (OverseerAutoScalingTriggerThread-99313032355577861-127.0.0.1:56264_solr-n_0000000000) [n:127.0.0.1:56264_solr    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 70518 ERROR (Thread[Thread-139,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:56264_solr    ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 71429 INFO  (jetty-closer-170-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5bbdbf9c{/solr,null,UNAVAILABLE}
   [junit4]   2> 71483 INFO  (jetty-closer-170-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2bdcc1c2{/solr,null,UNAVAILABLE}
   [junit4]   2> 71485 ERROR (SUITE-TestSolrCloudWithSecureImpersonation-seed#[93E0442371F0484A]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 71486 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[93E0442371F0484A]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:59021 59021
   [junit4]   2> 76544 INFO  (Thread-121) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:59021 59021
   [junit4]   2> 76544 WARN  (Thread-121) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/aliases.json
   [junit4]   2> 2 /solr/security.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 6 /solr/collections/forwardingCollection/state.json
   [junit4]   2> 4 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMKeyIdRoot
   [junit4]   2> 3 /solr/clusterstate.json
   [junit4]   2> 3 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMSeqNumRoot
   [junit4]   2> 3 /solr/clusterprops.json
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_1
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_3
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_2
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_4
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/live_nodes
   [junit4]   2> 3 /solr/collections
   [junit4]   2> 3 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot
   [junit4]   2>
   [junit4]   2> 76572 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[93E0442371F0484A]-worker) [    ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
   [junit4]   2> Jan 08, 2018 7:30:34 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> Jan 08, 2018 7:30:54 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
   [junit4]   2>    1) Thread[id=746, name=jetty-launcher-169-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
   [junit4]   2>         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
   [junit4]   2>         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
   [junit4]   2>         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
   [junit4]   2>         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
   [junit4]   2>         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
   [junit4]   2>         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
   [junit4]   2>         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
   [junit4]   2>         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
   [junit4]   2>         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
   [junit4]   2>         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
   [junit4]   2>         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2>         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> Jan 08, 2018 7:30:54 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=746, name=jetty-launcher-169-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
   [junit4]   2> 96752 ERROR (jetty-launcher-169-thread-2-EventThread) [n:127.0.0.1:38616_solr    ] o.a.c.f.i.CuratorFrameworkImpl Watcher exception
   [junit4]   2> java.lang.InterruptedException
   [junit4]   2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
   [junit4]   2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
   [junit4]   2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
   [junit4]   2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
   [junit4]   2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
   [junit4]   2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
   [junit4]   2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
   [junit4]   2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
   [junit4]   2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
   [junit4]   2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]   2> Jan 08, 2018 7:30:54 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=285, maxMBSortInHeap=7.653506893308355, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@13bb3578), locale=th-TH-u-nu-thai-x-lvariant-TH, timezone=Asia/Dhaka
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=42419912,total=132452352
   [junit4]   2> NOTE: All tests run in this JVM: [OpenExchangeRatesOrgProviderTest, TestManagedSchema, HdfsRestartWhileUpdatingTest, TestCollapseQParserPlugin, TestFastWriter, TestPartialUpdateDeduplication, TestZkChroot, TestFreeTextSuggestions, RemoteQueryErrorTest, CdcrReplicationDistributedZkTest, HdfsChaosMonkeyNothingIsSafeTest, TestSort, TestMergePolicyConfig, TestInitQParser, DocumentAnalysisRequestHandlerTest, PeerSyncTest, TestSolrCloudWithSecureImpersonation]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSolrCloudWithSecureImpersonation -Dtests.seed=93E0442371F0484A -Dtests.slow=true -Dtests.locale=th-TH-u-nu-thai-x-lvariant-TH -Dtests.timezone=Asia/Dhaka -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | TestSolrCloudWithSecureImpersonation (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
   [junit4]    >    1) Thread[id=746, name=jetty-launcher-169-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
   [junit4]    >         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
   [junit4]    >         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
   [junit4]    >         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
   [junit4]    >         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([93E0442371F0484A]:0)
   [junit4] Completed [27/766 (1!)] on J1 in 42.96s, 12 tests, 1 error <<< FAILURES!

[...truncated 50562 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1614 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1614/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory

Error Message:
Error from server at http://127.0.0.1:36617/solr: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:54319/solr within 10000 ms

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:36617/solr: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:54319/solr within 10000 ms
        at __randomizedtesting.SeedInfo.seed([284AD0DEDF26BAD0:45B67423656E45D7]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
        at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:267)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:     1) Thread[id=34733, name=jetty-launcher-6085-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)    2) Thread[id=34742, name=jetty-launcher-6085-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
   1) Thread[id=34733, name=jetty-launcher-6085-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
        at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
        at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
        at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
        at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
        at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   2) Thread[id=34742, name=jetty-launcher-6085-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
        at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
        at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
        at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
        at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
        at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
        at __randomizedtesting.SeedInfo.seed([284AD0DEDF26BAD0]:0)




Build Log:
[...truncated 11955 lines...]
   [junit4] Suite: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/init-core-data-001
   [junit4]   2> 525090 WARN  (SUITE-AutoscalingHistoryHandlerTest-seed#[284AD0DEDF26BAD0]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=5 numCloses=5
   [junit4]   2> 525090 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[284AD0DEDF26BAD0]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 525091 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[284AD0DEDF26BAD0]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 525091 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[284AD0DEDF26BAD0]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 525092 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[284AD0DEDF26BAD0]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001
   [junit4]   2> 525092 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[284AD0DEDF26BAD0]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 525092 INFO  (Thread-1895) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 525092 INFO  (Thread-1895) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 525094 ERROR (Thread-1895) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 525192 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[284AD0DEDF26BAD0]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:54319
   [junit4]   2> 525196 INFO  (zkConnectionManagerCallback-1117-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 525202 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x160d6f364b30000, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 525204 INFO  (jetty-launcher-1114-thread-1) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 525204 INFO  (jetty-launcher-1114-thread-3) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 525206 INFO  (jetty-launcher-1114-thread-2) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 525206 INFO  (jetty-launcher-1114-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6847402d{/solr,null,AVAILABLE}
   [junit4]   2> 525208 INFO  (jetty-launcher-1114-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@151f0d1c{/solr,null,AVAILABLE}
   [junit4]   2> 525214 INFO  (jetty-launcher-1114-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3b9197dd{HTTP/1.1,[http/1.1]}{127.0.0.1:36617}
   [junit4]   2> 525214 INFO  (jetty-launcher-1114-thread-3) [    ] o.e.j.s.Server Started @527608ms
   [junit4]   2> 525214 INFO  (jetty-launcher-1114-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36617}
   [junit4]   2> 525214 ERROR (jetty-launcher-1114-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 525214 INFO  (jetty-launcher-1114-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 525214 INFO  (jetty-launcher-1114-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 525214 INFO  (jetty-launcher-1114-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 525215 INFO  (jetty-launcher-1114-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-08T18:06:03.565Z
   [junit4]   2> 525217 INFO  (jetty-launcher-1114-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@384b4507{/solr,null,AVAILABLE}
   [junit4]   2> 525217 INFO  (jetty-launcher-1114-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@30c9e385{HTTP/1.1,[http/1.1]}{127.0.0.1:61612}
   [junit4]   2> 525217 INFO  (jetty-launcher-1114-thread-2) [    ] o.e.j.s.Server Started @527612ms
   [junit4]   2> 525218 INFO  (jetty-launcher-1114-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=61612}
   [junit4]   2> 525218 ERROR (jetty-launcher-1114-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 525218 INFO  (jetty-launcher-1114-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 525218 INFO  (jetty-launcher-1114-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 525218 INFO  (jetty-launcher-1114-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 525218 INFO  (jetty-launcher-1114-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-08T18:06:03.568Z
   [junit4]   2> 525225 INFO  (jetty-launcher-1114-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3bfbdcdb{HTTP/1.1,[http/1.1]}{127.0.0.1:47174}
   [junit4]   2> 525226 INFO  (jetty-launcher-1114-thread-1) [    ] o.e.j.s.Server Started @527620ms
   [junit4]   2> 525226 INFO  (jetty-launcher-1114-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=47174}
   [junit4]   2> 525226 ERROR (jetty-launcher-1114-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 525226 INFO  (jetty-launcher-1114-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 525226 INFO  (jetty-launcher-1114-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 525226 INFO  (jetty-launcher-1114-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 525226 INFO  (jetty-launcher-1114-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-08T18:06:03.576Z
   [junit4]   2> 525232 INFO  (zkConnectionManagerCallback-1121-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 525233 INFO  (zkConnectionManagerCallback-1119-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 525234 INFO  (jetty-launcher-1114-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 525235 INFO  (zkConnectionManagerCallback-1123-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 525236 INFO  (jetty-launcher-1114-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 525247 INFO  (jetty-launcher-1114-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 525266 INFO  (jetty-launcher-1114-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54319/solr
   [junit4]   2> 525266 INFO  (jetty-launcher-1114-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54319/solr
   [junit4]   2> 525268 INFO  (jetty-launcher-1114-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54319/solr
   [junit4]   2> 525272 INFO  (zkConnectionManagerCallback-1130-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 525272 INFO  (zkConnectionManagerCallback-1135-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 525281 INFO  (zkConnectionManagerCallback-1139-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 525282 INFO  (zkConnectionManagerCallback-1132-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 525283 INFO  (zkConnectionManagerCallback-1137-thread-1-processing-n:127.0.0.1:47174_solr) [n:127.0.0.1:47174_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 525288 INFO  (zkConnectionManagerCallback-1141-thread-1-processing-n:127.0.0.1:61612_solr) [n:127.0.0.1:61612_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 525463 INFO  (jetty-launcher-1114-thread-3) [n:127.0.0.1:36617_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 525464 INFO  (jetty-launcher-1114-thread-3) [n:127.0.0.1:36617_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36617_solr
   [junit4]   2> 525465 INFO  (jetty-launcher-1114-thread-3) [n:127.0.0.1:36617_solr    ] o.a.s.c.Overseer Overseer (id=99315532657000456-127.0.0.1:36617_solr-n_0000000000) starting
   [junit4]   2> 525466 INFO  (jetty-launcher-1114-thread-2) [n:127.0.0.1:61612_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 525468 INFO  (jetty-launcher-1114-thread-2) [n:127.0.0.1:61612_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61612_solr
   [junit4]   2> 525470 INFO  (zkCallback-1140-thread-1-processing-n:127.0.0.1:61612_solr) [n:127.0.0.1:61612_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 525470 INFO  (zkCallback-1138-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 525475 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 525476 DEBUG (OverseerAutoScalingTriggerThread-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 525477 INFO  (jetty-launcher-1114-thread-1) [n:127.0.0.1:47174_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 525477 DEBUG (OverseerAutoScalingTriggerThread-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 525478 DEBUG (OverseerAutoScalingTriggerThread-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:61612_solr]
   [junit4]   2> 525478 DEBUG (OverseerAutoScalingTriggerThread-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 525478 DEBUG (OverseerAutoScalingTriggerThread-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 525478 INFO  (jetty-launcher-1114-thread-3) [n:127.0.0.1:36617_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36617_solr
   [junit4]   2> 525479 INFO  (zkCallback-1140-thread-1-processing-n:127.0.0.1:61612_solr) [n:127.0.0.1:61612_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 525480 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:36617_solr"} current state version: 0
   [junit4]   2> 525480 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:36617_solr
   [junit4]   2> 525480 INFO  (zkCallback-1138-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 525481 DEBUG (OverseerAutoScalingTriggerThread-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 525481 DEBUG (OverseerAutoScalingTriggerThread-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 525482 DEBUG (ScheduledTrigger-3193-thread-1) [n:127.0.0.1:36617_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 525490 INFO  (zkCallback-1136-thread-1-processing-n:127.0.0.1:47174_solr) [n:127.0.0.1:47174_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 525491 INFO  (jetty-launcher-1114-thread-1) [n:127.0.0.1:47174_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 525493 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:47174_solr"} current state version: 0
   [junit4]   2> 525493 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:47174_solr
   [junit4]   2> 525493 INFO  (jetty-launcher-1114-thread-1) [n:127.0.0.1:47174_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47174_solr
   [junit4]   2> 525501 INFO  (zkCallback-1138-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 525506 INFO  (zkCallback-1140-thread-1-processing-n:127.0.0.1:61612_solr) [n:127.0.0.1:61612_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 525511 INFO  (zkCallback-1136-thread-1-processing-n:127.0.0.1:47174_solr) [n:127.0.0.1:47174_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 525622 INFO  (jetty-launcher-1114-thread-3) [n:127.0.0.1:36617_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36617.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a135f1e
   [junit4]   2> 525626 INFO  (jetty-launcher-1114-thread-2) [n:127.0.0.1:61612_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61612.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a135f1e
   [junit4]   2> 525626 INFO  (jetty-launcher-1114-thread-1) [n:127.0.0.1:47174_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47174.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a135f1e
   [junit4]   2> 525652 INFO  (jetty-launcher-1114-thread-1) [n:127.0.0.1:47174_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47174.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a135f1e
   [junit4]   2> 525652 INFO  (jetty-launcher-1114-thread-1) [n:127.0.0.1:47174_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47174.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a135f1e
   [junit4]   2> 525652 INFO  (jetty-launcher-1114-thread-3) [n:127.0.0.1:36617_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36617.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a135f1e
   [junit4]   2> 525652 INFO  (jetty-launcher-1114-thread-3) [n:127.0.0.1:36617_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36617.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a135f1e
   [junit4]   2> 525652 INFO  (jetty-launcher-1114-thread-2) [n:127.0.0.1:61612_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61612.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a135f1e
   [junit4]   2> 525652 INFO  (jetty-launcher-1114-thread-2) [n:127.0.0.1:61612_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61612.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a135f1e
   [junit4]   2> 525654 INFO  (jetty-launcher-1114-thread-3) [n:127.0.0.1:36617_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node2/.
   [junit4]   2> 525654 INFO  (jetty-launcher-1114-thread-2) [n:127.0.0.1:61612_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/.
   [junit4]   2> 525654 INFO  (jetty-launcher-1114-thread-1) [n:127.0.0.1:47174_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/.
   [junit4]   2> 525719 INFO  (zkConnectionManagerCallback-1149-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 525722 INFO  (zkConnectionManagerCallback-1153-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 525724 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[284AD0DEDF26BAD0]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 525725 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[284AD0DEDF26BAD0]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:54319/solr ready
   [junit4]   2> 525731 INFO  (qtp419901834-6415) [n:127.0.0.1:47174_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=3&name=AutoscalingHistoryHandlerTest_collection&nrtReplicas=3&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 525734 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.CreateCollectionCmd Create collection AutoscalingHistoryHandlerTest_collection
   [junit4]   2> 525734 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.CreateCollectionCmd Only _default config set found, using it.
   [junit4]   2> 525735 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/AutoscalingHistoryHandlerTest_collection/protwords.txt
   [junit4]   2> 525738 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/AutoscalingHistoryHandlerTest_collection/managed-schema
   [junit4]   2> 525740 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/AutoscalingHistoryHandlerTest_collection/solrconfig.xml
   [junit4]   2> 525743 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/AutoscalingHistoryHandlerTest_collection/synonyms.txt
   [junit4]   2> 525745 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/AutoscalingHistoryHandlerTest_collection/stopwords.txt
   [junit4]   2> 525747 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/params.json to /configs/AutoscalingHistoryHandlerTest_collection/params.json
   [junit4]   2> 525749 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/contractions_it.txt
   [junit4]   2> 525752 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stemdict_nl.txt
   [junit4]   2> 525755 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_no.txt
   [junit4]   2> 525757 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_hy.txt
   [junit4]   2> 525759 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/contractions_ca.txt
   [junit4]   2> 525762 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_id.txt
   [junit4]   2> 525764 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_ga.txt
   [junit4]   2> 525767 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_da.txt
   [junit4]   2> 525770 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_hi.txt
   [junit4]   2> 525773 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_ja.txt
   [junit4]   2> 525775 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_pt.txt
   [junit4]   2> 525777 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_hu.txt
   [junit4]   2> 525780 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_el.txt
   [junit4]   2> 525782 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_ru.txt
   [junit4]   2> 525784 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_tr.txt
   [junit4]   2> 525787 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_ar.txt
   [junit4]   2> 525789 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/userdict_ja.txt
   [junit4]   2> 525792 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_eu.txt
   [junit4]   2> 525794 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_it.txt
   [junit4]   2> 525797 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_cz.txt
   [junit4]   2> 525799 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stoptags_ja.txt
   [junit4]   2> 525802 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/contractions_fr.txt
   [junit4]   2> 525804 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_de.txt
   [junit4]   2> 525806 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_fa.txt
   [junit4]   2> 525809 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/contractions_ga.txt
   [junit4]   2> 525811 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_ca.txt
   [junit4]   2> 525813 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_nl.txt
   [junit4]   2> 525815 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_bg.txt
   [junit4]   2> 525817 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_lv.txt
   [junit4]   2> 525820 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_en.txt
   [junit4]   2> 525822 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_es.txt
   [junit4]   2> 525824 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_ro.txt
   [junit4]   2> 525827 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_th.txt
   [junit4]   2> 525829 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_fr.txt
   [junit4]   2> 525831 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_sv.txt
   [junit4]   2> 525833 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_fi.txt
   [junit4]   2> 525836 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/hyphenations_ga.txt
   [junit4]   2> 525838 INFO  (OverseerThreadFactory-3195-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_gl.txt
   [junit4]   2> 525846 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "name":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 0
   [junit4]   2> 525846 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.ClusterStateMutator building a new cName: AutoscalingHistoryHandlerTest_collection
   [junit4]   2> 525846 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json
   [junit4]   2> 525962 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:47174/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 525962 INFO  (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:47174/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 525966 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{}}
   [junit4]   2> 525966 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"http://127.0.0.1:47174/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 525969 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:36617/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 525969 INFO  (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:36617/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 525972 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"http://127.0.0.1:47174/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 525972 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"http://127.0.0.1:47174/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>       "base_url":"http://127.0.0.1:36617/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 525976 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:61612/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 525976 INFO  (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:61612/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 525979 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"http://127.0.0.1:47174/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>       "base_url":"http://127.0.0.1:36617/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 525979 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"http://127.0.0.1:47174/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node5":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>       "base_url":"http://127.0.0.1:36617/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node6":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>       "base_url":"http://127.0.0.1:61612/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 526080 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 0
   [junit4]   2> 526180 INFO  (qtp419901834-6412) [n:127.0.0.1:47174_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=AutoscalingHistoryHandlerTest_collection&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 526180 INFO  (qtp419901834-6412) [n:127.0.0.1:47174_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 526181 INFO  (qtp321081867-6434) [n:127.0.0.1:61612_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=AutoscalingHistoryHandlerTest_collection&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n4&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 526181 INFO  (qtp258177823-6430) [n:127.0.0.1:36617_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=AutoscalingHistoryHandlerTest_collection&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n2&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 526181 INFO  (qtp321081867-6434) [n:127.0.0.1:61612_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 526181 INFO  (qtp258177823-6430) [n:127.0.0.1:36617_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 526186 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:47174/solr",
   [junit4]   2>   "node_name":"127.0.0.1:47174_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 526187 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:47174/solr",
   [junit4]   2>   "node_name":"127.0.0.1:47174_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 526187 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node3:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1","base_url":"http://127.0.0.1:47174/solr","node_name":"127.0.0.1:47174_solr","state":"down","type":"NRT"}
   [junit4]   2> 526187 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"http://127.0.0.1:47174/solr",
   [junit4]   2>           "node_name":"127.0.0.1:47174_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>           "base_url":"http://127.0.0.1:36617/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>           "base_url":"http://127.0.0.1:61612/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 526187 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "core_node_name":"core_node6",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:61612/solr",
   [junit4]   2>   "node_name":"127.0.0.1:61612_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 526187 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>   "core_node_name":"core_node6",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:61612/solr",
   [junit4]   2>   "node_name":"127.0.0.1:61612_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 526187 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node6:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4","base_url":"http://127.0.0.1:61612/solr","node_name":"127.0.0.1:61612_solr","state":"down","type":"NRT"}
   [junit4]   2> 526187 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"http://127.0.0.1:47174/solr",
   [junit4]   2>           "node_name":"127.0.0.1:47174_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>           "base_url":"http://127.0.0.1:36617/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>           "base_url":"http://127.0.0.1:61612/solr",
   [junit4]   2>           "node_name":"127.0.0.1:61612_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 526187 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node5",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:36617/solr",
   [junit4]   2>   "node_name":"127.0.0.1:36617_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 526187 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node5",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:36617/solr",
   [junit4]   2>   "node_name":"127.0.0.1:36617_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 526187 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node5:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2","base_url":"http://127.0.0.1:36617/solr","node_name":"127.0.0.1:36617_solr","state":"down","type":"NRT"}
   [junit4]   2> 526187 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"http://127.0.0.1:47174/solr",
   [junit4]   2>           "node_name":"127.0.0.1:47174_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
   [junit4]   2>           "base_url":"http://127.0.0.1:36617/solr",
   [junit4]   2>           "node_name":"127.0.0.1:36617_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
   [junit4]   2>           "base_url":"http://127.0.0.1:61612/solr",
   [junit4]   2>           "node_name":"127.0.0.1:61612_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 526288 DEBUG (OverseerStateUpdate-99315532657000456-127.0.0.1:36617_solr-n_0000000000) [n:127.0.0.1:36617_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 1
   [junit4]   2> 526289 INFO  (zkCallback-1138-thread-1-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 526289 INFO  (zkCallback-1140-thread-1-processing-n:127.0.0.1:61612_solr) [n:127.0.0.1:61612_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 526289 INFO  (zkCallback-1136-thread-1-processing-n:127.0.0.1:47174_solr) [n:127.0.0.1:47174_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 526290 INFO  (zkCallback-1140-thread-2-processing-n:127.0.0.1:61612_solr) [n:127.0.0.1:61612_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 526290 INFO  (zkCallback-1138-thread-2-processing-n:127.0.0.1:36617_solr) [n:127.0.0.1:36617_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 526290 INFO  (zkCallback-1136-thread-2-processing-n:127.0.0.1:47174_solr) [n:127.0.0.1:47174_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 526492 DEBUG (ScheduledTrigger-3193-thread-1) [n:127.0.0.1:36617_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 527217 INFO  (qtp419901834-6412) [n:127.0.0.1:47174_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 527217 INFO  (qtp419901834-6412) [n:127.0.0.1:47174_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 527219 WARN  (qtp419901834-6412) [n:127.0.0.1:47174_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib
   [junit4]   2> 527219 WARN  (qtp419901834-6412) [n:127.0.0.1:47174_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 527220 INFO  (qtp321081867-6434) [n:127.0.0.1:61612_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 527221 INFO  (qtp321081867-6434) [n:127.0.0.1:61612_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 527221 WARN  (qtp419901834-6412) [n:127.0.0.1:47174_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib
   [junit4]   2> 527221 WARN  (qtp419901834-6412) [n:127.0.0.1:47174_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 527221 WARN  (qtp321081867-6434) [n:127.0.0.1:61612_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/extraction/lib
   [junit4]   2> 527222 WARN  (qtp419901834-6412) [n:127.0.0.1:47174_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/langid/lib
   [junit4]   2> 527222 WARN  (qtp321081867-6434) [n:127.0.0.1:61612_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
   [junit4]   2> 527222 WARN  (qtp419901834-6412) [n:127.0.0.1:47174_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 527222 WARN  (qtp321081867-6434) [n:127.0.0.1:61612_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/clustering/lib
   [junit4]   2> 527223 WARN  (qtp419901834-6412) [n:127.0.0.1:47174_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/velocity/lib
   [junit4]   2> 527223 WARN  (qtp321081867-6434) [n:127.0.0.1:61612_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
   [junit4]   2> 527223 WARN  (qtp419901834-6412) [n:127.0.0.1:47174_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 527223 WARN  (qtp321081867-6434) [n:127.0.0.1:61612_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/langid/lib
   [junit4]   2> 527223 INFO  (qtp258177823-6430) [n:127.0.0.1:36617_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 527223 WARN  (qtp321081867-6434) [n:127.0.0.1:61612_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
   [junit4]   2> 527223 INFO  (qtp258177823-6430) [n:127.0.0.1:36617_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 527223 WARN  (qtp321081867-6434) [n:127.0.0.1:61612_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/velocity/lib
   [junit4]   2> 527224 WARN  (qtp321081867-6434) [n:127.0.0.1:61612_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
   [junit4]   2> 527224 WARN  (qtp258177823-6430) [n:127.0.0.1:36617_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib
   [junit4]   2> 527224 WARN  (qtp258177823-6430) [n:127.0.0.1:36617_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 527224 WARN  (qtp258177823-6430) [n:127.0.0.1:36617_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib
   [junit4]   2> 527225 WARN  (qtp258177823-6430) [n:127.0.0.1:36617_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 527225 WARN  (qtp258177823-6430) [n:127.0.0.1:36617_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib
   [junit4]   2> 527225 WARN  (qtp258177823-6430) [n:127.0.0.1:36617_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_284AD0DEDF26BAD0-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
   [junit4]   2> 527225 WARN  (qtp258177823-6430) [n:127.0.0.1:36617_solr c:A

[...truncated too long message...]

stributedSpellCheckComponentTest, TestNumericTokenStream, AutoAddReplicasPlanActionTest, TestExactSharedStatsCache, TestReplicationHandler, MultiThreadedOCPTest, SubstringBytesRefFilterTest, DeleteShardTest, UtilsToolTest, SpatialHeatmapFacetsTest, TestPseudoReturnFields, TestHashQParserPlugin, LegacyCloudClusterPropTest, TestDynamicFieldCollectionResource, SuggesterTest, DistributedFacetPivotSmallAdvancedTest, AutoscalingHistoryHandlerTest, TestExclusionRuleCollectionAccess, TestSimDistributedQueue, TestJsonFacetRefinement, TestComputePlanAction, ForceLeaderTest, LargeFieldTest, TemplateUpdateProcessorTest, TestShortCircuitedRequests, InfoHandlerTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestNodeLostTrigger, SpellCheckComponentTest, TestCoreDiscovery, SolrCloudExampleTest, SolrCoreMetricManagerTest, TimeZoneUtilsTest, UUIDFieldTest, SystemLogListenerTest, TestTrieFacet, BigEndianAscendingWordDeserializerTest, FullHLLTest, IndexBasedSpellCheckerTest, TestLegacyTerms, TestPerFieldSimilarityWithDefaultOverride, TestLeaderInitiatedRecoveryThread, IndexSchemaTest, UpdateLogTest, ShardRoutingTest, JavabinLoaderTest, TestHighFrequencyDictionaryFactory, TestTolerantUpdateProcessorCloud, TestRuleBasedAuthorizationPlugin, ComputePlanActionTest, TestBulkSchemaConcurrent, OverseerCollectionConfigSetProcessorTest, RankQueryTest, TestStressRecovery, DirectUpdateHandlerTest, SimpleMLTQParserTest, TestExecutePlanAction, AsyncCallRequestStatusResponseTest, TestSearcherReuse, CollectionTooManyReplicasTest, SliceStateTest, TestManagedStopFilterFactory, TestSchemaSimilarityResource, BinaryUpdateRequestHandlerTest, RecoveryAfterSoftCommitTest, DeleteNodeTest, TestAddFieldRealTimeGet, DistanceFunctionTest, ChaosMonkeyNothingIsSafeTest, TestFieldTypeCollectionResource, TestMinMaxOnMultiValuedField, TestRequestForwarding, DistributedVersionInfoTest, BlobRepositoryCloudTest, TestFuzzyAnalyzedSuggestions, TestConfigsApi, TestJoin, CheckHdfsIndexTest, TestNumericTerms32, NodeLostTriggerTest, BasicAuthStandaloneTest, ResourceLoaderTest, MinimalSchemaTest, TestHalfAndHalfDocValues, SolrGangliaReporterTest, TestQuerySenderNoQuery, TestSolrConfigHandlerConcurrent, RequestLoggingTest, QueryResultKeyTest, TestSSLRandomization, TestFilteredDocIdSet, MetricTriggerTest, SolrInfoBeanTest, JSONWriterTest, SpatialFilterTest, HLLUtilTest, LeaderInitiatedRecoveryOnCommitTest, TestRawTransformer, TestTolerantSearch, TestCoreBackup, TestConfigOverlay, TestRecovery, TestRecoveryHdfs, HdfsRecoveryZkTest, TestFiltering, BlockDirectoryTest, CustomHighlightComponentTest, TestRealTimeGet, TestMissingGroups, TestPhraseSuggestions, HttpPartitionTest, TestCorePropertiesReload, MetricUtilsTest, TestCollectionsAPIViaSolrCloudCluster, TestCloudInspectUtil, TestGraphMLResponseWriter, CollectionReloadTest, DateFieldTest, SortByFunctionTest, CloneFieldUpdateProcessorFactoryTest, TriLevelCompositeIdRoutingTest, TestSurroundQueryParser, TestDFRSimilarityFactory, ZkControllerTest, SolrTestCaseJ4Test, DistributedTermsComponentTest, TestCustomSort, TestNumericRangeQuery32, TestBulkSchemaAPI, HighlighterTest, CleanupOldIndexTest, TestConfigSetProperties, SuggestComponentTest, TestSolrCoreParser, SuggesterFSTTest, RequestHandlersTest, TestUpdate, ManagedSchemaRoundRobinCloudTest, TestLRUStatsCache, SyncSliceTest, DistributedFacetPivotWhiteBoxTest, TestShardHandlerFactory, ReplaceNodeTest, ConfigSetsAPITest, UniqFieldsUpdateProcessorFactoryTest, TestIntervalFaceting, MissingSegmentRecoveryTest, WordBreakSolrSpellCheckerTest, SolrCmdDistributorTest, TestCodecSupport, TestDelegationWithHadoopAuth, TestLFUCache, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, OpenCloseCoreStressTest, OverseerTest, RecoveryZkTest, ClusterStateUpdateTest, LeaderElectionTest, ZkSolrClientTest, TestDistributedSearch, StatsComponentTest, DistributedQueryElevationComponentTest, SolrIndexSplitterTest, SimplePostToolTest, SpellCheckCollatorTest, CoreAdminHandlerTest, SuggesterWFSTTest, PolyFieldTest, SchemaVersionSpecificBehaviorTest, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, StatelessScriptUpdateProcessorFactoryTest, CacheHeaderTest, DisMaxRequestHandlerTest, TestWriterPerf, DocumentAnalysisRequestHandlerTest, TestQueryTypes, TestOmitPositions, TermsComponentTest, TestIndexingPerformance, FieldAnalysisRequestHandlerTest, RegexBoostProcessorTest, TestJmxIntegration, UpdateRequestProcessorFactoryTest, QueryParsingTest, TestPHPSerializedResponseWriter, PingRequestHandlerTest, TestSolrCoreProperties, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, SpellPossibilityIteratorTest, TestXIncludeConfig, TestPerFieldSimilarity, TestIBSimilarityFactory, TestFastWriter, ScriptEngineTest, URLClassifyProcessorTest, PrimUtilsTest, SpellingQueryConverterTest, RAMDirectoryFactoryTest, TestLRUCache, TestDocumentBuilder, SystemInfoHandlerTest, CircularListTest, CursorPagingTest, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, AddReplicaTest, AssignBackwardCompatibilityTest, ConnectionManagerTest, CreateCollectionCleanupTest, DistribDocExpirationUpdateProcessorTest, MigrateRouteKeyTest, MoveReplicaHDFSTest, OverseerRolesTest, OverseerTaskQueueTest, ShardSplitTest, TestPullReplica, TestRandomFlRTGCloud, TestRebalanceLeaders, TestRequestStatusCollectionAPI, TestSolrCloudWithKerberosAlt, TestSolrCloudWithSecureImpersonation]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSolrCloudWithSecureImpersonation -Dtests.seed=284AD0DEDF26BAD0 -Dtests.slow=true -Dtests.locale=ar-QA -Dtests.timezone=Europe/Gibraltar -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J0 | TestSolrCloudWithSecureImpersonation (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
   [junit4]    >    1) Thread[id=34733, name=jetty-launcher-6085-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
   [junit4]    >         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
   [junit4]    >         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
   [junit4]    >         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
   [junit4]    >         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]    >    2) Thread[id=34742, name=jetty-launcher-6085-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
   [junit4]    >         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
   [junit4]    >         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
   [junit4]    >         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
   [junit4]    >         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
   [junit4]    >         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
   [junit4]    >         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([284AD0DEDF26BAD0]:0)
   [junit4] Completed [568/766 (2!)] on J0 in 43.42s, 12 tests, 1 error <<< FAILURES!

[...truncated 48760 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1615 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1615/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

9 tests failed.
FAILED:  org.apache.solr.client.solrj.request.SchemaTest.deletingAFieldThatDoesntExistInTheSchemaShouldFail

Error Message:
Error from server at http://127.0.0.1:65189/solr/collection1: error processing commands

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException: Error from server at http://127.0.0.1:65189/solr/collection1: error processing commands
        at __randomizedtesting.SeedInfo.seed([1EA27B6BB7092F7:38055E817110C0ED]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException.create(HttpSolrClient.java:829)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:620)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.client.solrj.request.SchemaTest.deletingAFieldThatDoesntExistInTheSchemaShouldFail(SchemaTest.java:332)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.client.solrj.request.SchemaTest.addDynamicFieldShouldntBeCalledTwiceWithTheSameName

Error Message:
Error from server at http://127.0.0.1:47187/solr/collection1: error processing commands

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException: Error from server at http://127.0.0.1:47187/solr/collection1: error processing commands
        at __randomizedtesting.SeedInfo.seed([1EA27B6BB7092F7:4B1A6890636127EA]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException.create(HttpSolrClient.java:829)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:620)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.client.solrj.request.SchemaTest.addDynamicFieldShouldntBeCalledTwiceWithTheSameName(SchemaTest.java:417)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.client.solrj.request.SchemaTest.deletingADynamicFieldThatDoesntExistInTheSchemaShouldFail

Error Message:
Error from server at http://127.0.0.1:39897/solr/collection1: error processing commands

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException: Error from server at http://127.0.0.1:39897/solr/collection1: error processing commands
        at __randomizedtesting.SeedInfo.seed([1EA27B6BB7092F7:8519245570ABE638]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException.create(HttpSolrClient.java:829)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:620)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.client.solrj.request.SchemaTest.deletingADynamicFieldThatDoesntExistInTheSchemaShouldFail(SchemaTest.java:458)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.client.solrj.request.SchemaTest.addFieldTypeShouldntBeCalledTwiceWithTheSameName

Error Message:
Error from server at http://127.0.0.1:62329/solr/collection1: error processing commands

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException: Error from server at http://127.0.0.1:62329/solr/collection1: error processing commands
        at __randomizedtesting.SeedInfo.seed([1EA27B6BB7092F7:C40BD5A6450D805F]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException.create(HttpSolrClient.java:829)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:620)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.client.solrj.request.SchemaTest.addFieldTypeShouldntBeCalledTwiceWithTheSameName(SchemaTest.java:649)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.client.solrj.request.SchemaTest.copyFieldsShouldFailWhenOneOfTheFieldsDoesntExistInTheSchema

Error Message:
Error from server at http://127.0.0.1:58079/solr/collection1: error processing commands

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException: Error from server at http://127.0.0.1:58079/solr/collection1: error processing commands
        at __randomizedtesting.SeedInfo.seed([1EA27B6BB7092F7:2E2E00168F78EA9D]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException.create(HttpSolrClient.java:829)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:620)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.client.solrj.request.SchemaTest.copyFieldsShouldFailWhenOneOfTheFieldsDoesntExistInTheSchema(SchemaTest.java:806)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.client.solrj.request.SchemaTest.deleteCopyFieldShouldFailWhenOneOfTheFieldsDoesntExistInTheSchema

Error Message:
Error from server at http://127.0.0.1:54020/solr/collection1: error processing commands

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException: Error from server at http://127.0.0.1:54020/solr/collection1: error processing commands
        at __randomizedtesting.SeedInfo.seed([1EA27B6BB7092F7:C4772A40CC6F7289]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException.create(HttpSolrClient.java:829)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:620)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.client.solrj.request.SchemaTest.deleteCopyFieldShouldFailWhenOneOfTheFieldsDoesntExistInTheSchema(SchemaTest.java:841)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.client.solrj.request.SchemaTest.addFieldShouldntBeCalledTwiceWithTheSameName

Error Message:
Error from server at http://127.0.0.1:36404/solr/collection1: error processing commands

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException: Error from server at http://127.0.0.1:36404/solr/collection1: error processing commands
        at __randomizedtesting.SeedInfo.seed([1EA27B6BB7092F7:D0994FEC0FC8725B]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException.create(HttpSolrClient.java:829)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:620)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.client.solrj.request.SchemaTest.addFieldShouldntBeCalledTwiceWithTheSameName(SchemaTest.java:293)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.client.solrj.request.SchemaTest.deletingAFieldTypeThatDoesntExistInTheSchemaShouldFail

Error Message:
Error from server at http://127.0.0.1:46549/solr/collection1: error processing commands

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException: Error from server at http://127.0.0.1:46549/solr/collection1: error processing commands
        at __randomizedtesting.SeedInfo.seed([1EA27B6BB7092F7:7ADC166CF26ABE22]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException.create(HttpSolrClient.java:829)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:620)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.client.solrj.request.SchemaTest.deletingAFieldTypeThatDoesntExistInTheSchemaShouldFail(SchemaTest.java:694)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:     1) Thread[id=29698, name=jetty-launcher-6272-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)    2) Thread[id=29710, name=jetty-launcher-6272-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
   1) Thread[id=29698, name=jetty-launcher-6272-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
        at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
        at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
        at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
        at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
        at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   2) Thread[id=29710, name=jetty-launcher-6272-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
        at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
        at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
        at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
        at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
        at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
        at __randomizedtesting.SeedInfo.seed([5C97D2F15D0FD796]:0)




Build Log:
[...truncated 13502 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
   [junit4]   2> 3358281 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[5C97D2F15D0FD796]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_5C97D2F15D0FD796-001/init-core-data-001
   [junit4]   2> 3358281 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[5C97D2F15D0FD796]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 3358282 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[5C97D2F15D0FD796]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 3358311 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[5C97D2F15D0FD796]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_5C97D2F15D0FD796-001/tempDir-001
   [junit4]   2> 3358311 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[5C97D2F15D0FD796]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3358312 INFO  (Thread-6116) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3358312 INFO  (Thread-6116) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3358314 ERROR (Thread-6116) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 3358412 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[5C97D2F15D0FD796]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:51487
   [junit4]   2> 3358414 INFO  (zkConnectionManagerCallback-6275-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3358421 INFO  (jetty-launcher-6272-thread-2) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 3358422 INFO  (jetty-launcher-6272-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4013e306{/solr,null,AVAILABLE}
   [junit4]   2> 3358422 INFO  (jetty-launcher-6272-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1c07bb19{SSL,[ssl, http/1.1]}{127.0.0.1:45971}
   [junit4]   2> 3358422 INFO  (jetty-launcher-6272-thread-2) [    ] o.e.j.s.Server Started @3362663ms
   [junit4]   2> 3358422 INFO  (jetty-launcher-6272-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45971}
   [junit4]   2> 3358422 ERROR (jetty-launcher-6272-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3358423 INFO  (jetty-launcher-6272-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 3358423 INFO  (jetty-launcher-6272-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3358423 INFO  (jetty-launcher-6272-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3358423 INFO  (jetty-launcher-6272-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-09T06:19:10.640Z
   [junit4]   2> 3358425 INFO  (jetty-launcher-6272-thread-1) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 3358425 INFO  (jetty-launcher-6272-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1a8c2c07{/solr,null,AVAILABLE}
   [junit4]   2> 3358426 INFO  (jetty-launcher-6272-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5b28494a{SSL,[ssl, http/1.1]}{127.0.0.1:64978}
   [junit4]   2> 3358426 INFO  (jetty-launcher-6272-thread-1) [    ] o.e.j.s.Server Started @3362667ms
   [junit4]   2> 3358426 INFO  (jetty-launcher-6272-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=64978}
   [junit4]   2> 3358426 ERROR (jetty-launcher-6272-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3358426 INFO  (jetty-launcher-6272-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 3358426 INFO  (jetty-launcher-6272-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3358426 INFO  (jetty-launcher-6272-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3358426 INFO  (jetty-launcher-6272-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-09T06:19:10.643Z
   [junit4]   2> 3358429 INFO  (zkConnectionManagerCallback-6279-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3358429 INFO  (zkConnectionManagerCallback-6277-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3358430 INFO  (jetty-launcher-6272-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3358430 INFO  (jetty-launcher-6272-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3358435 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x160d99295420002, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 3358435 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x160d99295420001, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 3358436 INFO  (jetty-launcher-6272-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51487/solr
   [junit4]   2> 3358437 INFO  (jetty-launcher-6272-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51487/solr
   [junit4]   2> 3358439 INFO  (zkConnectionManagerCallback-6287-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3358439 INFO  (zkConnectionManagerCallback-6284-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3358441 INFO  (zkConnectionManagerCallback-6289-thread-1-processing-n:127.0.0.1:45971_solr) [n:127.0.0.1:45971_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3358443 INFO  (zkConnectionManagerCallback-6291-thread-1-processing-n:127.0.0.1:64978_solr) [n:127.0.0.1:64978_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3358507 INFO  (jetty-launcher-6272-thread-1) [n:127.0.0.1:64978_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3358508 INFO  (jetty-launcher-6272-thread-1) [n:127.0.0.1:64978_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:64978_solr
   [junit4]   2> 3358508 INFO  (jetty-launcher-6272-thread-1) [n:127.0.0.1:64978_solr    ] o.a.s.c.Overseer Overseer (id=99318415394734086-127.0.0.1:64978_solr-n_0000000000) starting
   [junit4]   2> 3358516 INFO  (jetty-launcher-6272-thread-1) [n:127.0.0.1:64978_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64978_solr
   [junit4]   2> 3358558 INFO  (zkCallback-6290-thread-1-processing-n:127.0.0.1:64978_solr) [n:127.0.0.1:64978_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3358584 INFO  (jetty-launcher-6272-thread-1) [n:127.0.0.1:64978_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
   [junit4]   2> 3358585 WARN  (jetty-launcher-6272-thread-1) [n:127.0.0.1:64978_solr    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 3358585 INFO  (jetty-launcher-6272-thread-1) [n:127.0.0.1:64978_solr    ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
   [junit4]   2> 3358586 INFO  (jetty-launcher-6272-thread-1) [n:127.0.0.1:64978_solr    ] o.a.c.f.i.CuratorFrameworkImpl Starting
   [junit4]   2> 3358588 INFO  (jetty-launcher-6272-thread-1-EventThread) [n:127.0.0.1:64978_solr    ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
   [junit4]   2> 3358604 WARN  (jetty-launcher-6272-thread-1) [n:127.0.0.1:64978_solr    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 3358662 INFO  (jetty-launcher-6272-thread-2) [n:127.0.0.1:45971_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3358664 INFO  (jetty-launcher-6272-thread-2) [n:127.0.0.1:45971_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3358666 INFO  (jetty-launcher-6272-thread-2) [n:127.0.0.1:45971_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45971_solr
   [junit4]   2> 3358687 INFO  (zkCallback-6290-thread-2-processing-n:127.0.0.1:64978_solr) [n:127.0.0.1:64978_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3358690 INFO  (jetty-launcher-6272-thread-2) [n:127.0.0.1:45971_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
   [junit4]   2> 3358690 WARN  (jetty-launcher-6272-thread-2) [n:127.0.0.1:45971_solr    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 3358691 INFO  (jetty-launcher-6272-thread-2) [n:127.0.0.1:45971_solr    ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
   [junit4]   2> 3358691 INFO  (zkCallback-6288-thread-1-processing-n:127.0.0.1:45971_solr) [n:127.0.0.1:45971_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3358691 INFO  (jetty-launcher-6272-thread-2) [n:127.0.0.1:45971_solr    ] o.a.c.f.i.CuratorFrameworkImpl Starting
   [junit4]   2> 3358694 INFO  (jetty-launcher-6272-thread-2-EventThread) [n:127.0.0.1:45971_solr    ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
   [junit4]   2> 3358697 INFO  (jetty-launcher-6272-thread-1) [n:127.0.0.1:64978_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64978.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@37ae9baa
   [junit4]   2> 3358711 INFO  (jetty-launcher-6272-thread-1) [n:127.0.0.1:64978_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64978.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@37ae9baa
   [junit4]   2> 3358711 INFO  (jetty-launcher-6272-thread-1) [n:127.0.0.1:64978_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64978.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@37ae9baa
   [junit4]   2> 3358712 WARN  (jetty-launcher-6272-thread-2) [n:127.0.0.1:45971_solr    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 3358724 INFO  (jetty-launcher-6272-thread-1) [n:127.0.0.1:64978_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_5C97D2F15D0FD796-001/tempDir-001/node1/.
   [junit4]   2> 3358763 INFO  (jetty-launcher-6272-thread-2) [n:127.0.0.1:45971_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45971.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@37ae9baa
   [junit4]   2> 3358769 INFO  (jetty-launcher-6272-thread-2) [n:127.0.0.1:45971_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45971.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@37ae9baa
   [junit4]   2> 3358769 INFO  (jetty-launcher-6272-thread-2) [n:127.0.0.1:45971_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45971.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@37ae9baa
   [junit4]   2> 3358769 INFO  (jetty-launcher-6272-thread-2) [n:127.0.0.1:45971_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_5C97D2F15D0FD796-001/tempDir-001/node2/.
   [junit4]   2> 3358812 INFO  (zkConnectionManagerCallback-6297-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3358816 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x160d99295420009, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 3358830 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[5C97D2F15D0FD796]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidGroup
   [junit4]   2> 3358869 WARN  (qtp1886024879-29657) [    ] o.a.h.s.ShellBasedUnixGroupsMapping got exception trying to get groups for user bar: id: invalid user name: "bar"
   [junit4]   2> id: invalid user name: "bar"
   [junit4]   2>
   [junit4]   2> 3358893 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[5C97D2F15D0FD796]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bogusGroup&p=bogusGroup&t=test&e=1515514751064&s=ETPVeTsDCFI0QDmdDKtTfnRszWo="; Path=/; Domain=127.0.0.1; Expires=mar, 09-ene-2018 16:19:11 GMT; Secure; HttpOnly". Invalid 'expires' attribute: mar, 09-ene-2018 16:19:11 GMT
   [junit4]   2> 3358893 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[5C97D2F15D0FD796]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidGroup
   [junit4]   2> 3358919 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[5C97D2F15D0FD796]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigHosts
   [junit4]   2> 3358923 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[5C97D2F15D0FD796]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noHosts&p=noHosts&t=test&e=1515514751137&s=P0Yv5FDeEeEwL/kId6lneOZyc6s="; Path=/; Domain=127.0.0.1; Expires=mar, 09-ene-2018 16:19:11 GMT; Secure; HttpOnly". Invalid 'expires' attribute: mar, 09-ene-2018 16:19:11 GMT
   [junit4]   2> 3358924 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[5C97D2F15D0FD796]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigHosts
   [junit4]   2> 3358948 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[5C97D2F15D0FD796]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidProxyUser
   [junit4]   2> 3358951 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[5C97D2F15D0FD796]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bar&p=bar&t=test&e=1515514751166&s=p8oJYs45s03bwTZk15HHcZAiH3E="; Path=/; Domain=127.0.0.1; Expires=mar, 09-ene-2018 16:19:11 GMT; Secure; HttpOnly". Invalid 'expires' attribute: mar, 09-ene-2018 16:19:11 GMT
   [junit4]   2> 3358951 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[5C97D2F15D0FD796]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidProxyUser
   [junit4]   2> 3358974 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[5C97D2F15D0FD796]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidRemote
   [junit4]   2> 3358982 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[5C97D2F15D0FD796]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1515514751192&s=doDlPRUKKWAvUrB1BsFBY6zQj8U="; Path=/; Domain=127.0.0.1; Expires=mar, 09-ene-2018 16:19:11 GMT; Secure; HttpOnly". Invalid 'expires' attribute: mar, 09-ene-2018 16:19:11 GMT
   [junit4]   2> 3358982 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[5C97D2F15D0FD796]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidRemote
   [junit4]   2> 3359002 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[5C97D2F15D0FD796]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigGroups
   [junit4]   2> 3359004 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[5C97D2F15D0FD796]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noGroups&p=noGroups&t=test&e=1515514751220&s=+PN9K8N2vyXcRPFJ853/cToCcCk="; Path=/; Domain=127.0.0.1; Expires=mar, 09-ene-2018 16:19:11 GMT; Secure; HttpOnly". Invalid 'expires' attribute: mar, 09-ene-2018 16:19:11 GMT
   [junit4]   2> 3359004 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[5C97D2F15D0FD796]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigGroups
   [junit4]   2> 3359019 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[5C97D2F15D0FD796]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testForwarding
   [junit4]   2> 3359024 INFO  (zkConnectionManagerCallback-6300-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3359490 INFO  (zkConnectionManagerCallback-6303-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3359491 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[5C97D2F15D0FD796]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 3359492 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[5C97D2F15D0FD796]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51487/solr ready
   [junit4]   2> 3359496 INFO  (qtp1886024879-29650) [n:127.0.0.1:45971_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 3359498 INFO  (OverseerThreadFactory-13044-thread-1-processing-n:127.0.0.1:64978_solr) [n:127.0.0.1:64978_solr    ] o.a.s.c.CreateCollectionCmd Create collection forwardingCollection
   [junit4]   2> 3359611 INFO  (OverseerStateUpdate-99318415394734086-127.0.0.1:64978_solr-n_0000000000) [n:127.0.0.1:64978_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"forwardingCollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"forwardingCollection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:64978/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 3359829 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 3359830 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3359934 INFO  (zkCallback-6290-thread-1-processing-n:127.0.0.1:64978_solr) [n:127.0.0.1:64978_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3359935 INFO  (zkCallback-6290-thread-2-processing-n:127.0.0.1:64978_solr) [n:127.0.0.1:64978_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3360848 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 3360865 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema [forwardingCollection_shard1_replica_n1] Schema name=test
   [junit4]   2> 3360973 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 3360992 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'forwardingCollection_shard1_replica_n1' using configuration from collection forwardingCollection, trusted=true
   [junit4]   2> 3360992 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64978.solr.core.forwardingCollection.shard1.replica_n1' (registry 'solr.core.forwardingCollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@37ae9baa
   [junit4]   2> 3360993 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 3360993 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [[forwardingCollection_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_5C97D2F15D0FD796-001/tempDir-001/node1/forwardingCollection_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_5C97D2F15D0FD796-001/tempDir-001/node1/./forwardingCollection_shard1_replica_n1/data/]
   [junit4]   2> 3360994 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=79.76171875, floorSegmentMB=1.6806640625, forceMergeDeletesPctAllowed=3.8347192687374476, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.709164582439119
   [junit4]   2> 3361045 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 3361045 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3361046 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3361046 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3361047 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=93.9833984375, floorSegmentMB=0.787109375, forceMergeDeletesPctAllowed=5.507355054161941, segmentsPerTier=38.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 3361047 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5de6c1d2[forwardingCollection_shard1_replica_n1] main]
   [junit4]   2> 3361050 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 3361053 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 3361067 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 3361067 INFO  (searcherExecutor-13049-thread-1-processing-n:127.0.0.1:64978_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 3361067 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1589094649203523584
   [junit4]   2> 3361069 INFO  (searcherExecutor-13049-thread-1-processing-n:127.0.0.1:64978_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 3361069 INFO  (searcherExecutor-13049-thread-1-processing-n:127.0.0.1:64978_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 3361069 INFO  (searcherExecutor-13049-thread-1-processing-n:127.0.0.1:64978_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 3361069 INFO  (searcherExecutor-13049-thread-1-processing-n:127.0.0.1:64978_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 3361070 INFO  (searcherExecutor-13049-thread-1-processing-n:127.0.0.1:64978_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 3361070 INFO  (searcherExecutor-13049-thread-1-processing-n:127.0.0.1:64978_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 3361070 INFO  (searcherExecutor-13049-thread-1-processing-n:127.0.0.1:64978_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 3361070 INFO  (searcherExecutor-13049-thread-1-processing-n:127.0.0.1:64978_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 3361071 INFO  (searcherExecutor-13049-thread-1-processing-n:127.0.0.1:64978_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] Registered new searcher Searcher@5de6c1d2[forwardingCollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3361074 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3361074 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3361074 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:64978/solr/forwardingCollection_shard1_replica_n1/
   [junit4]   2> 3361074 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3361074 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:64978/solr/forwardingCollection_shard1_replica_n1/ has no replicas
   [junit4]   2> 3361074 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 3361079 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:64978/solr/forwardingCollection_shard1_replica_n1/ shard1
   [junit4]   2> 3361181 INFO  (zkCallback-6290-thread-2-processing-n:127.0.0.1:64978_solr) [n:127.0.0.1:64978_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3361181 INFO  (zkCallback-6290-thread-1-processing-n:127.0.0.1:64978_solr) [n:127.0.0.1:64978_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3361231 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3361234 INFO  (qtp788146827-29667) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1405
   [junit4]   2> 3361234 WARN  (httpShardExecutor-13040-thread-1-processing-n:127.0.0.1:64978_solr [https:////127.0.0.1:64978//solr] https:////127.0.0.1:64978//solr) [n:127.0.0.1:64978_solr    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1515514752046&s=TIMkNJrWWqY9wDpfENgOLdfOGHw="; Path=/; Domain=127.0.0.1; Expires=mar, 09-ene-2018 16:19:12 GMT; Secure; HttpOnly". Invalid 'expires' attribute: mar, 09-ene-2018 16:19:12 GMT
   [junit4]   2> 3361238 INFO  (qtp1886024879-29650) [n:127.0.0.1:45971_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 3361335 INFO  (zkCallback-6290-thread-1-processing-n:127.0.0.1:64978_solr) [n:127.0.0.1:64978_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3361336 INFO  (zkCallback-6290-thread-3-processing-n:127.0.0.1:64978_solr) [n:127.0.0.1:64978_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3361502 INFO  (OverseerCollectionConfigSetProcessor-99318415394734086-127.0.0.1:64978_solr-n_0000000000) [n:127.0.0.1:64978_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 3362240 INFO  (qtp1886024879-29650) [n:127.0.0.1:45971_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=2744
   [junit4]   2> 3362241 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[5C97D2F15D0FD796]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1515514751713&s=k5wxz1dfecOkbkleHGvIh6ZXyD8="; Path=/; Domain=127.0.0.1; Expires=mar, 09-ene-2018 16:19:11 GMT; Secure; HttpOnly". Invalid 'expires' attribute: mar, 09-ene-2018 16:19:11 GMT
   [junit4]   2> 3362241 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[5C97D2F15D0FD796]) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: forwardingCollection failOnTimeout:true timeout (sec):100
   [junit4]   2> 3362242 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[5C97D2F15D0FD796]) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: forwardingCollection
   [junit4]   2> 3362246 INFO  (qtp1886024879-29658) [n:127.0.0.1:45971_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3362252 INFO  (qtp788146827-29664) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 3362252 WARN  (qtp1886024879-29658) [n:127.0.0.1:45971_solr    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1515514754468&s=N6/BdYFO9zgeXQXmNZ+EADfo/lY="; Path=/; Domain=127.0.0.1; Expires=mar, 09-ene-2018 16:19:14 GMT; Secure; HttpOnly". Invalid 'expires' attribute: mar, 09-ene-2018 16:19:14 GMT
   [junit4]   2> 3362253 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[5C97D2F15D0FD796]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1515514754462&s=KX7uGBjHDl/Ty5OVcBf8QH/nSww="; Path=/; Domain=127.0.0.1; Expires=mar, 09-ene-2018 16:19:14 GMT; Secure; HttpOnly". Invalid 'expires' attribute: mar, 09-ene-2018 16:19:14 GMT
   [junit4]   2> 3362253 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[5C97D2F15D0FD796]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1515514754468&s=N6/BdYFO9zgeXQXmNZ+EADfo/lY="; Path=/; Domain=127.0.0.1; Expires=mar, 09-ene-2018 16:19:14 GMT; Secure; HttpOnly". Invalid 'expires' attribute: mar, 09-ene-2018 16:19:14 GMT
   [junit4]   2> 3362257 INFO  (qtp788146827-29665) [n:127.0.0.1:64978_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 3362257 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[5C97D2F15D0FD796]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1515514754473&s=Tfxqjwx0SVCBj3h1aJ2JQnUF0ho="; Path=/; Domain=127.0.0.1; Expires=mar, 09-ene-2018 16:19:14 GMT; Secure; HttpOnly". Invalid 'expires' attribute: mar, 09-ene-2018 16:19:14 GMT
   [junit4]   2> 3362257 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[5C97D2F15D0FD796]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testForwarding
   [junit4]   2> 3362279 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[5C97D2F15D0FD796]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateAnyHostAnyUser
   [junit4]   2> 3362281 INFO  (qtp1886024879-29659) [n:127.0.0.1:45971_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 3362282 INFO  (qtp1886024879-29659) [n:127.0.0.1:45971_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 3362282 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[5C97D2F15D0FD796]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostAnyUser&p=anyHostAnyUser&t=test&e=1515514754498&s=0w1wIfqr754Xg56zfNUKFDfIaYs="; Path=/; Domain=127.0.0.1; Expires=mar, 09-ene-2018 16:19:14 GMT; Secure; HttpOnly". Invalid 'expires' attribute: mar, 09-ene-2018 16:19:14 GMT
   [junit4]   2> 3362282 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[5C97D2F15D0FD796]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateAnyHostAnyUser
   [junit4]   2> 3362306 INFO  (TEST-TestSolrCloudWithSecureImpe

[...truncated too long message...]

/tempDir-103/collection1/conf/managed-schema
   [junit4]   2> 150561 INFO  (coreLoadExecutor-976-thread-1) [    x:collection1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema, renamed the non-managed schema /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.request.SchemaTest_1EA27B6BB7092F7-001/tempDir-103/collection1/conf/schema.xml to /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.request.SchemaTest_1EA27B6BB7092F7-001/tempDir-103/collection1/conf/schema.xml.bak
   [junit4]   2> 150561 INFO  (coreLoadExecutor-976-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.request.SchemaTest_1EA27B6BB7092F7-001/tempDir-103/collection1, trusted=true
   [junit4]   2> 150561 INFO  (coreLoadExecutor-976-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ddce8c8
   [junit4]   2> 150561 INFO  (coreLoadExecutor-976-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 150561 INFO  (coreLoadExecutor-976-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.request.SchemaTest_1EA27B6BB7092F7-001/tempDir-103/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.request.SchemaTest_1EA27B6BB7092F7-001/tempDir-105/cores/core/data/]
   [junit4]   2> 150608 WARN  (coreLoadExecutor-976-thread-1) [    x:collection1] o.a.s.c.RequestHandlers no default request handler is registered (either '/select' or 'standard')
   [junit4]   2> 150608 INFO  (coreLoadExecutor-976-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 150608 INFO  (coreLoadExecutor-976-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 150610 INFO  (coreLoadExecutor-976-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@83155e2[collection1] main]
   [junit4]   2> 150610 INFO  (coreLoadExecutor-976-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.request.SchemaTest_1EA27B6BB7092F7-001/tempDir-103/collection1/conf
   [junit4]   2> 150611 INFO  (coreLoadExecutor-976-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 150612 INFO  (searcherExecutor-977-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@83155e2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 151112 INFO  (TEST-SchemaTest.testReplaceDynamicFieldAccuracy-seed#[1EA27B6BB7092F7]) [    ] o.a.s.SolrJettyTestBase Jetty Assigned Port#42402
   [junit4]   2> 151117 INFO  (qtp2065632685-2877) [    x:collection1] o.a.s.s.ManagedIndexSchema Upgraded to managed schema at /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.request.SchemaTest_1EA27B6BB7092F7-001/tempDir-103/collection1/conf/managed-schema
   [junit4]   2> 151118 INFO  (qtp2065632685-2877) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/schema params={wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 151123 INFO  (qtp2065632685-2881) [    x:collection1] o.a.s.s.ManagedIndexSchema Upgraded to managed schema at /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.request.SchemaTest_1EA27B6BB7092F7-001/tempDir-103/collection1/conf/managed-schema
   [junit4]   2> 151124 INFO  (qtp2065632685-2881) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/schema params={wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 151125 INFO  (qtp2065632685-2883) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/schema/dynamicfields/*_replace params={wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 151126 INFO  (TEST-SchemaTest.testReplaceDynamicFieldAccuracy-seed#[1EA27B6BB7092F7]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@6f2d5d23{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 151126 INFO  (TEST-SchemaTest.testReplaceDynamicFieldAccuracy-seed#[1EA27B6BB7092F7]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=487350800
   [junit4]   2> 151126 INFO  (TEST-SchemaTest.testReplaceDynamicFieldAccuracy-seed#[1EA27B6BB7092F7]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 151126 INFO  (TEST-SchemaTest.testReplaceDynamicFieldAccuracy-seed#[1EA27B6BB7092F7]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@dbe53e8: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@76413e0b
   [junit4]   2> 151132 INFO  (TEST-SchemaTest.testReplaceDynamicFieldAccuracy-seed#[1EA27B6BB7092F7]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 151132 INFO  (TEST-SchemaTest.testReplaceDynamicFieldAccuracy-seed#[1EA27B6BB7092F7]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@46c2fe99: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@11095d69
   [junit4]   2> 151137 INFO  (TEST-SchemaTest.testReplaceDynamicFieldAccuracy-seed#[1EA27B6BB7092F7]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 151139 INFO  (TEST-SchemaTest.testReplaceDynamicFieldAccuracy-seed#[1EA27B6BB7092F7]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@102d5cd0: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@64b532b9
   [junit4]   2> 151145 INFO  (coreCloseExecutor-981-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@18239a23
   [junit4]   2> 151145 INFO  (coreCloseExecutor-981-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=404986403
   [junit4]   2> 151145 INFO  (coreCloseExecutor-981-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4428848: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@71908f50
   [junit4]   2> 151162 INFO  (TEST-SchemaTest.testReplaceDynamicFieldAccuracy-seed#[1EA27B6BB7092F7]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1306d852{/solr,null,UNAVAILABLE}
   [junit4]   2> 151164 INFO  (TEST-SchemaTest.testReplaceDynamicFieldAccuracy-seed#[1EA27B6BB7092F7]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testReplaceDynamicFieldAccuracy
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.request.SchemaTest_1EA27B6BB7092F7-001
   [junit4]   2> NOTE: test params are: codec=Lucene70, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@47e22c74), locale=es-NI, timezone=America/Indiana/Winamac
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=1,free=310058816,total=465567744
   [junit4]   2> NOTE: All tests run in this JVM: [JsonValidatorTest, CloudSolrClientTest, LargeVolumeEmbeddedTest, JdbcDriverTest, TestSuggesterResponse, TestConfigSetAdminRequest, HyperbolicTangentEvaluatorTest, LargeVolumeBinaryJettyTest, FieldValueEvaluatorTest, LargeVolumeJettyTest, TestFastInputStream, ClientUtilsTest, SolrExceptionTest, SolrPingTest, GraphExpressionTest, ArcCosineEvaluatorTest, MergeIndexesEmbeddedTest, AscEvaluatorTest, RegressionEvaluatorTest, QueryResponseTest, NaturalLogEvaluatorTest, SolrExampleStreamingBinaryTest, LessThanEqualToEvaluatorTest, JDBCStreamTest, GreaterThanEqualToEvaluatorTest, AndEvaluatorTest, AbsoluteValueEvaluatorTest, BasicHttpSolrClientTest, FacetFieldTest, OperationsTest, RawValueEvaluatorTest, HttpSolrClientConPoolTest, ConcurrentUpdateSolrClientTest, CommonParamsTest, TestV2Request, CloudSolrClientBuilderTest, CloudSolrClientCacheTest, LBHttpSolrClientBuilderTest, SelectWithEvaluatorsTest, StreamExpressionToExpessionTest, StreamExpressionToExplanationTest, StreamingTest, AppendEvaluatorTest, ArcSineEvaluatorTest, ArcTangentEvaluatorTest, ArrayEvaluatorTest, CoalesceEvaluatorTest, CorrelationEvaluatorTest, CubedRootEvaluatorTest, CumulativeProbabilityEvaluatorTest, DivideEvaluatorTest, EqualToEvaluatorTest, FloorEvaluatorTest, GreaterThanEvaluatorTest, HyperbolicCosineEvaluatorTest, LengthEvaluatorTest, LessThanEvaluatorTest, ModuloEvaluatorTest, NormalizeEvaluatorTest, OrEvaluatorTest, RecursiveEvaluatorTest, RoundEvaluatorTest, SquareRootEvaluatorTest, SubtractEvaluatorTest, TangentEvaluatorTest, UniformDistributionEvaluatorTest, UuidEvaluatorTest, StreamExpressionParserTest, ConcatOperationTest, SchemaTest]
   [junit4] Completed [132/146 (1!)] on J1 in 19.39s, 35 tests, 8 errors <<< FAILURES!

[...truncated 47211 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1616 - Failure!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1616/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

3 tests failed.
FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigReplication

Error Message:
expected:<497> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<497> but was:<0>
        at __randomizedtesting.SeedInfo.seed([9E2335ACCEE6B1C3:8A6B6EF9EDE10CDD]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigReplication(TestReplicationHandler.java:513)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.test

Error Message:
Error from server at http://127.0.0.1:36049/solr/myalias_2017-10-23_shard1_replica_n2: Unexpected type of routeKey: null

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:36049/solr/myalias_2017-10-23_shard1_replica_n2: Unexpected type of routeKey: null
        at __randomizedtesting.SeedInfo.seed([9E2335ACCEE6B1C3:16770A76601ADC3B]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:560)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1013)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
        at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
        at org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.test(TimeRoutedAliasUpdateProcessorTest.java:141)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:     1) Thread[id=20285, name=jetty-launcher-5286-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)    2) Thread[id=20289, name=jetty-launcher-5286-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
   1) Thread[id=20285, name=jetty-launcher-5286-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
        at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
        at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
        at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
        at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
        at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
   2) Thread[id=20289, name=jetty-launcher-5286-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
        at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
        at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
        at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
        at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
        at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
        at __randomizedtesting.SeedInfo.seed([9E2335ACCEE6B1C3]:0)




Build Log:
[...truncated 12249 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> 1083975 INFO  (SUITE-TestReplicationHandler-seed#[9E2335ACCEE6B1C3]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/init-core-data-001
   [junit4]   2> 1083976 WARN  (SUITE-TestReplicationHandler-seed#[9E2335ACCEE6B1C3]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=23 numCloses=23
   [junit4]   2> 1083976 INFO  (SUITE-TestReplicationHandler-seed#[9E2335ACCEE6B1C3]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1083977 INFO  (SUITE-TestReplicationHandler-seed#[9E2335ACCEE6B1C3]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
   [junit4]   2> 1083979 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.SolrTestCaseJ4 ###Starting doTestReplicateAfterStartupWithNoActivity
   [junit4]   2> 1083979 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-001/collection1
   [junit4]   2> 1083982 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1083982 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5f2f15cb{/solr,null,AVAILABLE}
   [junit4]   2> 1083983 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5927cc18{HTTP/1.1,[http/1.1]}{127.0.0.1:37862}
   [junit4]   2> 1083983 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.e.j.s.Server Started @1086380ms
   [junit4]   2> 1083983 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=37862}
   [junit4]   2> 1083983 ERROR (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1083983 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1083983 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 1083983 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1083983 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-09T18:23:40.739Z
   [junit4]   2> 1083985 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-001/solr.xml
   [junit4]   2> 1083988 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1083988 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1083989 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3f2f954f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1084019 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f2f954f
   [junit4]   2> 1084025 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f2f954f
   [junit4]   2> 1084026 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f2f954f
   [junit4]   2> 1084027 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-001/.
   [junit4]   2> 1084027 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 1084028 INFO  (coreLoadExecutor-3270-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1084040 INFO  (coreLoadExecutor-3270-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1084044 INFO  (coreLoadExecutor-3270-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1084045 INFO  (coreLoadExecutor-3270-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
   [junit4]   2> 1084045 INFO  (coreLoadExecutor-3270-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-001/./collection1, trusted=true
   [junit4]   2> 1084046 INFO  (coreLoadExecutor-3270-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f2f954f
   [junit4]   2> 1084046 INFO  (coreLoadExecutor-3270-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1084046 INFO  (coreLoadExecutor-3270-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-001/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 1084047 INFO  (coreLoadExecutor-3270-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=40, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1084089 INFO  (coreLoadExecutor-3270-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1084089 INFO  (coreLoadExecutor-3270-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1084090 INFO  (coreLoadExecutor-3270-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=62.17578125, floorSegmentMB=2.103515625, forceMergeDeletesPctAllowed=29.369632755202037, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3854610459825236
   [junit4]   2> 1084090 INFO  (coreLoadExecutor-3270-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1ee43597[collection1] main]
   [junit4]   2> 1084090 INFO  (coreLoadExecutor-3270-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-001/collection1/conf
   [junit4]   2> 1084091 INFO  (coreLoadExecutor-3270-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 1084091 INFO  (coreLoadExecutor-3270-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1084092 INFO  (searcherExecutor-3271-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1ee43597[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1084092 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-002/collection1
   [junit4]   2> 1084094 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1084096 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@777f615b{/solr,null,AVAILABLE}
   [junit4]   2> 1084096 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@625dbd66{HTTP/1.1,[http/1.1]}{127.0.0.1:60149}
   [junit4]   2> 1084096 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.e.j.s.Server Started @1086494ms
   [junit4]   2> 1084096 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=60149}
   [junit4]   2> 1084096 ERROR (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1084096 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1084096 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 1084096 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1084096 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-09T18:23:40.852Z
   [junit4]   2> 1084096 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-002/solr.xml
   [junit4]   2> 1084100 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1084100 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1084101 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3f2f954f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1084128 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f2f954f
   [junit4]   2> 1084136 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f2f954f
   [junit4]   2> 1084136 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f2f954f
   [junit4]   2> 1084137 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-002/.
   [junit4]   2> 1084137 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 1084138 INFO  (coreLoadExecutor-3280-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1084147 INFO  (coreLoadExecutor-3280-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1084151 INFO  (coreLoadExecutor-3280-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1084152 INFO  (coreLoadExecutor-3280-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
   [junit4]   2> 1084152 INFO  (coreLoadExecutor-3280-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-002/./collection1, trusted=true
   [junit4]   2> 1084153 INFO  (coreLoadExecutor-3280-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f2f954f
   [junit4]   2> 1084153 INFO  (coreLoadExecutor-3280-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1084153 INFO  (coreLoadExecutor-3280-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-002/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-002/./collection1/data/]
   [junit4]   2> 1084154 INFO  (coreLoadExecutor-3280-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=40, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1084190 INFO  (coreLoadExecutor-3280-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1084190 INFO  (coreLoadExecutor-3280-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1084190 INFO  (coreLoadExecutor-3280-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=62.17578125, floorSegmentMB=2.103515625, forceMergeDeletesPctAllowed=29.369632755202037, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3854610459825236
   [junit4]   2> 1084191 INFO  (coreLoadExecutor-3280-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@162af863[collection1] main]
   [junit4]   2> 1084191 INFO  (coreLoadExecutor-3280-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-002/collection1/conf
   [junit4]   2> 1084192 INFO  (coreLoadExecutor-3280-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 1084192 INFO  (coreLoadExecutor-3280-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1084193 INFO  (searcherExecutor-3281-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@162af863[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1084193 INFO  (qtp964879185-14037) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 1084194 INFO  (indexFetcher-3285-thread-1) [    x:collection1] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 1084194 INFO  (indexFetcher-3285-thread-1) [    x:collection1] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 1084194 INFO  (indexFetcher-3285-thread-1) [    x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 1084194 INFO  (indexFetcher-3285-thread-1) [    x:collection1] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 1084194 INFO  (indexFetcher-3285-thread-1) [    x:collection1] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
   [junit4]   2> 1084194 INFO  (indexFetcher-3285-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1084194 INFO  (indexFetcher-3285-thread-1) [    x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@7cfcbf96 commitCommandVersion:0
   [junit4]   2> 1084196 INFO  (indexFetcher-3285-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@152fa3f3[collection1] main]
   [junit4]   2> 1084197 INFO  (indexFetcher-3285-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1084196 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@625dbd66{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1084197 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=97332005
   [junit4]   2> 1084198 INFO  (searcherExecutor-3281-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@152fa3f3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1084198 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1084198 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@329c0ffb: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@326f73a4
   [junit4]   2> 1084202 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1084202 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@46a45c6d: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@366af0d8
   [junit4]   2> 1084205 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1084205 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@295e3a00: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@7fd0630e
   [junit4]   2> 1084206 INFO  (coreCloseExecutor-3286-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@42e11c3a
   [junit4]   2> 1084206 INFO  (coreCloseExecutor-3286-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1122049082
   [junit4]   2> 1084206 INFO  (coreCloseExecutor-3286-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1066a8dd: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@85d8dff
   [junit4]   2> 1084223 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@777f615b{/solr,null,UNAVAILABLE}
   [junit4]   2> 1084225 INFO  (qtp964879185-14040) [    x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 1084225 INFO  (qtp964879185-14040) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:*} 0 0
   [junit4]   2> 1084226 INFO  (qtp964879185-14043) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1084226 INFO  (qtp964879185-14043) [    x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@4c80ab0a commitCommandVersion:0
   [junit4]   2> 1084227 INFO  (qtp964879185-14043) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@37a56d1[collection1] main]
   [junit4]   2> 1084227 INFO  (qtp964879185-14043) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1084228 INFO  (searcherExecutor-3271-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@37a56d1[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1084228 INFO  (qtp964879185-14043) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 2
   [junit4]   2> 1084229 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@5927cc18{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1084229 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1062148871
   [junit4]   2> 1084229 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1084230 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4c658cff: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@22fe8c37
   [junit4]   2> 1084230 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1084230 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@568f2bac: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@366af0d8
   [junit4]   2> 1084231 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1084231 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6f151c0c: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@7fd0630e
   [junit4]   2> 1084233 INFO  (coreCloseExecutor-3288-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@17bc1c6f
   [junit4]   2> 1084233 INFO  (coreCloseExecutor-3288-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=398204015
   [junit4]   2> 1084233 INFO  (coreCloseExecutor-3288-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@788cd237: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@19b626f5
   [junit4]   2> 1084237 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5f2f15cb{/solr,null,UNAVAILABLE}
   [junit4]   2> 1084240 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1084241 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6b14ab50{/solr,null,AVAILABLE}
   [junit4]   2> 1084242 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@24c26af0{HTTP/1.1,[http/1.1]}{127.0.0.1:61096}
   [junit4]   2> 1084242 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.e.j.s.Server Started @1086640ms
   [junit4]   2> 1084242 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=61096}
   [junit4]   2> 1084242 ERROR (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1084243 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1084243 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 1084243 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1084243 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-09T18:23:40.999Z
   [junit4]   2> 1084243 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-001/solr.xml
   [junit4]   2> 1084245 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1084245 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1084246 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3f2f954f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1084273 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f2f954f
   [junit4]   2> 1084279 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f2f954f
   [junit4]   2> 1084279 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f2f954f
   [junit4]   2> 1084280 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-001/.
   [junit4]   2> 1084280 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[9E2335ACCEE6B1C3]) [    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 1084281 INFO  (coreLoadExecutor-3295-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1084291 INFO  (coreLoadExecutor-3295-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1084295 INFO  (coreLoadExecutor-3295-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1084297 INFO  (coreLoadExecutor-3295-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
   [junit4]   2> 1084297 INFO  (coreLoadExecutor-3295-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-001/./collection1, trusted=true
   [junit4]   2> 1084297 INFO  (coreLoadExecutor-3295-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f2f954f
   [junit4]   2> 1084297 INFO  (coreLoadExecutor-3295-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1084297 INFO  (coreLoadExecutor-3295-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-001/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 1084299 INFO  (coreLoadExecutor-3295-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=48, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1084334 INFO  (coreLoadExecutor-3295-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1084334 INFO  (coreLoadExecutor-3295-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1084335 INFO  (coreLoadExecutor-3295-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=27, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1084335 INFO  (coreLoadExecutor-3295-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@48ec160f[collection1] main]
   [junit4]   2> 1084335 INFO  (coreLoadExecutor-3295-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_9E2335ACCEE6B1C3-001/solr-instance-001/collection1/conf
   [junit4]   2> 1084336 INFO  (coreLoadExecutor-3295-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml
   [junit4]   2> 1084336 INFO  (coreLoadExecutor-3295-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1084336 INFO  (searcherExecutor-3296-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@48ec160f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1084339 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 0
   [junit4]   2> 1084340 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 0
   [junit4]   2> 1084340 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
   [junit4]   2> 1084341 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[3]} 0 0
   [junit4]   2> 1084341 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[4]} 0 0
   [junit4]   2> 1084342 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[5]} 0 0
   [junit4]   2> 1084342 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[6]} 0 0
   [junit4]   2> 1084343 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[7]} 0 0
   [junit4]   2> 1084343 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[8]} 0 0
   [junit4]   2> 1084343 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[9]} 0 0
   [junit4]   2> 1084344 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[10]} 0 0
   [junit4]   2> 1084345 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[11]} 0 0
   [junit4]   2> 1084345 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[12]} 0 0
   [junit4]   2> 1084345 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[13]} 0 0
   [junit4]   2> 1084345 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[14]} 0 0
   [junit4]   2> 1084345 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[15]} 0 0
   [junit4]   2> 1084346 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[16]} 0 0
   [junit4]   2> 1084346 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[17]} 0 0
   [junit4]   2> 1084346 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[18]} 0 0
   [junit4]   2> 1084346 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[19]} 0 0
   [junit4]   2> 1084346 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[20]} 0 0
   [junit4]   2> 1084347 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[21]} 0 0
   [junit4]   2> 1084347 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[22]} 0 0
   [junit4]   2> 1084347 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[23]} 0 0
   [junit4]   2> 1084347 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[24]} 0 0
   [junit4]   2> 1084348 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[25]} 0 0
   [junit4]   2> 1084348 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[26]} 0 0
   [junit4]   2> 1084349 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[27]} 0 0
   [junit4]   2> 1084349 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[28]} 0 0
   [junit4]   2> 1084350 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[29]} 0 0
   [junit4]   2> 1084350 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[30]} 0 0
   [junit4]   2> 1084350 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[31]} 0 0
   [junit4]   2> 1084351 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[32]} 0 0
   [junit4]   2> 1084351 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[33]} 0 0
   [junit4]   2> 1084352 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[34]} 0 0
   [junit4]   2> 1084352 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[35]} 0 0
   [junit4]   2> 1084352 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[36]} 0 0
   [junit4]   2> 1084353 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[37]} 0 0
   [junit4]   2> 1084353 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[38]} 0 0
   [junit4]   2> 1084353 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[39]} 0 0
   [junit4]   2> 1084354 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[40]} 0 0
   [junit4]   2> 1084354 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[41]} 0 0
   [junit4]   2> 1084354 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[42]} 0 0
   [junit4]   2> 1084355 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[43]} 0 0
   [junit4]   2> 1084355 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[44]} 0 0
   [junit4]   2> 1084355 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[45]} 0 0
   [junit4]   2> 1084355 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[46]} 0 0
   [junit4]   2> 1084356 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[47]} 0 0
   [junit4]   2> 1084356 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[48]} 0 0
   [junit4]   2> 1084357 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[49]} 0 0
   [junit4]   2> 1084357 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[50]} 0 0
   [junit4]   2> 1084357 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[51]} 0 0
   [junit4]   2> 1084358 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[52]} 0 0
   [junit4]   2> 1084358 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[53]} 0 0
   [junit4]   2> 1084358 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[54]} 0 0
   [junit4]   2> 1084359 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[55]} 0 0
   [junit4]   2> 1084359 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[56]} 0 0
   [junit4]   2> 1084360 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[57]} 0 0
   [junit4]   2> 1084360 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[58]} 0 0
   [junit4]   2> 1084360 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[59]} 0 0
   [junit4]   2> 1084361 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[60]} 0 0
   [junit4]   2> 1084361 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[61]} 0 0
   [junit4]   2> 1084362 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[62]} 0 0
   [junit4]   2> 1084362 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[63]} 0 0
   [junit4]   2> 1084362 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[64]} 0 0
   [junit4]   2> 1084363 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[65]} 0 0
   [junit4]   2> 1084363 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[66]} 0 0
   [junit4]   2> 1084364 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[67]} 0 0
   [junit4]   2> 1084364 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[68]} 0 0
   [junit4]   2> 1084364 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[69]} 0 0
   [junit4]   2> 1084365 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[70]} 0 0
   [junit4]   2> 1084365 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[71]} 0 0
   [junit4]   2> 1084366 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[72]} 0 0
   [junit4]   2> 1084366 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[73]} 0 0
   [junit4]   2> 1084366 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[74]} 0 0
   [junit4]   2> 1084367 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[75]} 0 0
   [junit4]   2> 1084367 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[76]} 0 0
   [junit4]   2> 1084367 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[77]} 0 0
   [junit4]   2> 1084368 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[78]} 0 0
   [junit4]   2> 1084368 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[79]} 0 0
   [junit4]   2> 1084368 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[80]} 0 0
   [junit4]   2> 1084368 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[81]} 0 0
   [junit4]   2> 1084368 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[82]} 0 0
   [junit4]   2> 1084368 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[83]} 0 0
   [junit4]   2> 1084368 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[84]} 0 0
   [junit4]   2> 1084368 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[85]} 0 0
   [junit4]   2> 1084372 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[86]} 0 0
   [junit4]   2> 1084373 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[87]} 0 0
   [junit4]   2> 1084373 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[88]} 0 0
   [junit4]   2> 1084374 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[89]} 0 0
   [junit4]   2> 1084374 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[90]} 0 0
   [junit4]   2> 1084374 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[91]} 0 0
   [junit4]   2> 1084375 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[92]} 0 0
   [junit4]   2> 1084375 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[93]} 0 0
   [junit4]   2> 1084376 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[94]} 0 0
   [junit4]   2> 1084376 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[95]} 0 0
   [junit4]   2> 1084376 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[96]} 0 0
   [junit4]   2> 1084377 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[97]} 0 0
   [junit4]   2> 1084377 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[98]} 0 0
   [junit4]   2> 1084377 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[99]} 0 0
   [junit4]   2> 1084378 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[100]} 0 0
   [junit4]   2> 1084378 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[101]} 0 0
   [junit4]   2> 1084378 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[102]} 0 0
   [junit4]   2> 1084379 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[103]} 0 0
   [junit4]   2> 1084379 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[104]} 0 0
   [junit4]   2> 1084380 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[105]} 0 0
   [junit4]   2> 1084380 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[106]} 0 0
   [junit4]   2> 1084381 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[107]} 0 0
   [junit4]   2> 1084381 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[108]} 0 0
   [junit4]   2> 1084381 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[109]} 0 0
   [junit4]   2> 1084382 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[110]} 0 0
   [junit4]   2> 1084382 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[111]} 0 0
   [junit4]   2> 1084382 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[112]} 0 0
   [junit4]   2> 1084383 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[113]} 0 0
   [junit4]   2> 1084383 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[114]} 0 0
   [junit4]   2> 1084383 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[115]} 0 0
   [junit4]   2> 1084383 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[116]} 0 0
   [junit4]   2> 1084383 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[117]} 0 0
   [junit4]   2> 1084383 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[118]} 0 0
   [junit4]   2> 1084384 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[119]} 0 0
   [junit4]   2> 1084384 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[120]} 0 0
   [junit4]   2> 1084385 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[121]} 0 0
   [junit4]   2> 1084385 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[122]} 0 0
   [junit4]   2> 1084386 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[123]} 0 0
   [junit4]   2> 1084386 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[124]} 0 0
   [junit4]   2> 1084386 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[125]} 0 0
   [junit4]   2> 1084387 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[126]} 0 0
   [junit4]   2> 1084387 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[127]} 0 0
   [junit4]   2> 1084387 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[128]} 0 0
   [junit4]   2> 1084388 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[129]} 0 0
   [junit4]   2> 1084388 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[130]} 0 0
   [junit4]   2> 1084389 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[131]} 0 0
   [junit4]   2> 1084389 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[132]} 0 0
   [junit4]   2> 1084389 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[133]} 0 0
   [junit4]   2> 1084390 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[134]} 0 0
   [junit4]   2> 1084390 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[135]} 0 0
   [junit4]   2> 1084390 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[136]} 0 0
   [junit4]   2> 1084391 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[137]} 0 0
   [junit4]   2> 1084391 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[138]} 0 0
   [junit4]   2> 1084391 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[139]} 0 0
   [junit4]   2> 1084391 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[140]} 0 0
   [junit4]   2> 1084392 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[141]} 0 0
   [junit4]   2> 1084392 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[142]} 0 0
   [junit4]   2> 1084393 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[143]} 0 0
   [junit4]   2> 1084393 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[144]} 0 0
   [junit4]   2> 1084393 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[145]} 0 0
   [junit4]   2> 1084394 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[146]} 0 0
   [junit4]   2> 1084394 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[147]} 0 0
   [junit4]   2> 1084394 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[148]} 0 0
   [junit4]   2> 1084395 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[149]} 0 0
   [junit4]   2> 1084395 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[150]} 0 0
   [junit4]   2> 1084395 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[151]} 0 0
   [junit4]   2> 1084396 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[152]} 0 0
   [junit4]   2> 1084396 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[153]} 0 0
   [junit4]   2> 1084396 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[154]} 0 0
   [junit4]   2> 1084397 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[155]} 0 0
   [junit4]   2> 1084397 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[156]} 0 0
   [junit4]   2> 1084398 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[157]} 0 0
   [junit4]   2> 1084398 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[158]} 0 0
   [junit4]   2> 1084398 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[159]} 0 0
   [junit4]   2> 1084399 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[160]} 0 0
   [junit4]   2> 1084399 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[161]} 0 0
   [junit4]   2> 1084400 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[162]} 0 0
   [junit4]   2> 1084400 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[163]} 0 0
   [junit4]   2> 1084400 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[164]} 0 0
   [junit4]   2> 1084401 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[165]} 0 0
   [junit4]   2> 1084401 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[166]} 0 0
   [junit4]   2> 1084402 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[167]} 0 0
   [junit4]   2> 1084402 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[168]} 0 0
   [junit4]   2> 1084402 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[169]} 0 0
   [junit4]   2> 1084403 INFO  (qtp1815896551-14081) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[170]} 0 0
   [junit4]   2> 1084403 INFO  (qtp1815896551-14076) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[171]} 0 0
   [junit4]   2> 1084403 INFO  (qtp1815896551-14082) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[172]} 0 0
   [junit4]   2> 1084404 INFO  (qtp1815896551-14079) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[173]} 0 0
   [junit4]   2> 1084404 INFO  (qtp1815896551-14078) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[174]} 0 0
   [junit4]   2> 1084405 INFO  (qtp1815896551-14080) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[175]} 0 0
   [junit4]   2> 1084405 INFO  (qtp1815896551-14083) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[176]} 0 0
   [junit4]   2> 1084405 INFO  (qtp1815896551-14081

[...truncated too long message...]

etStream(StreamExpressionParser.parse("facet("
 [ecj-lint]                                                         +   "collection1, "
 [ecj-lint]                                                         +   "q=\"*:*\", "
 [ecj-lint]                                                         +   "buckets=\"a_s\", "
 [ecj-lint]                                                         +   "bucketSorts=\"sum(a_i) asc\", "
 [ecj-lint]                                                         +   "bucketSizeLimit=100, "
 [ecj-lint]                                                         +   "sum(a_i), sum(a_f), "
 [ecj-lint]                                                         +   "min(a_i), min(a_f), "
 [ecj-lint]                                                         +   "max(a_i), max(a_f), "
 [ecj-lint]                                                         +   "avg(a_i), avg(a_f), "
 [ecj-lint]                                                         +   "count(*)"
 [ecj-lint]                                                         + ")"), factory);
 [ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'stream' is never closed
 [ecj-lint] ----------
 [ecj-lint] 31. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/solrj/src/test/org/apache/solr/client/solrj/io/stream/StreamExpressionToExplanationTest.java (at line 250)
 [ecj-lint] stream = new JDBCStream(StreamExpressionParser.parse("jdbc(connection=\"jdbc:hsqldb:mem:.\", sql=\"select PEOPLE.ID, PEOPLE.NAME, COUNTRIES.COUNTRY_NAME from PEOPLE inner join COUNTRIES on PEOPLE.COUNTRY_CODE = COUNTRIES.CODE order by PEOPLE.ID\", sort=\"ID asc\")"), factory);
 [ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'stream' is never closed
 [ecj-lint] ----------
 [ecj-lint] 32. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/solrj/src/test/org/apache/solr/client/solrj/io/stream/StreamExpressionToExplanationTest.java (at line 263)
 [ecj-lint] stream = new IntersectStream(StreamExpressionParser.parse("intersect("
 [ecj-lint]                               + "search(collection1, q=\"id:(0 3 4)\", fl=\"id,a_s,a_i,a_f\", sort=\"a_f asc, a_s asc\"),"
 [ecj-lint]                               + "search(collection1, q=\"id:(1 2)\", fl=\"id,a_s,a_i,a_f\", sort=\"a_f asc, a_s asc\"),"
 [ecj-lint]                               + "on=\"a_f, a_s\")"), factory);
 [ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'stream' is never closed
 [ecj-lint] ----------
 [ecj-lint] 33. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/solrj/src/test/org/apache/solr/client/solrj/io/stream/StreamExpressionToExplanationTest.java (at line 279)
 [ecj-lint] stream = new ComplementStream(StreamExpressionParser.parse("complement("
 [ecj-lint]                               + "search(collection1, q=\"id:(0 3 4)\", fl=\"id,a_s,a_i,a_f\", sort=\"a_f asc, a_s asc\"),"
 [ecj-lint]                               + "search(collection1, q=\"id:(1 2)\", fl=\"id,a_s,a_i,a_f\", sort=\"a_f asc, a_s asc\"),"
 [ecj-lint]                               + "on=\"a_f, a_s\")"), factory);
 [ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'stream' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 34. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/solrj/src/test/org/apache/solr/client/solrj/io/stream/StreamingTest.java (at line 159)
 [ecj-lint] CloudSolrStream stream = new CloudSolrStream("", "collection1", sParams);
 [ecj-lint]                ^^^^^^
 [ecj-lint] Resource leak: 'stream' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 35. ERROR in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/solrj/src/test/org/apache/solr/client/solrj/request/SchemaTest.java (at line 31)
 [ecj-lint] import org.apache.solr.api.ApiBag;
 [ecj-lint]       ^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.solr.api.ApiBag is never used
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 36. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/solrj/src/test/org/apache/solr/client/solrj/response/NoOpResponseParserTest.java (at line 52)
 [ecj-lint] return new SolrResourceLoader().openResource("solrj/sampleRangeFacetResponse.xml");
 [ecj-lint]       ^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 37. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/solrj/src/test/org/apache/solr/client/solrj/response/QueryResponseTest.java (at line 46)
 [ecj-lint] InputStream is = new SolrResourceLoader().openResource("solrj/sampleRangeFacetResponse.xml");
 [ecj-lint]                 ^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
 [ecj-lint] ----------
 [ecj-lint] 38. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/solrj/src/test/org/apache/solr/client/solrj/response/QueryResponseTest.java (at line 103)
 [ecj-lint] InputStream is = new SolrResourceLoader().openResource("solrj/sampleGroupResponse.xml");
 [ecj-lint]                 ^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
 [ecj-lint] ----------
 [ecj-lint] 39. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/solrj/src/test/org/apache/solr/client/solrj/response/QueryResponseTest.java (at line 205)
 [ecj-lint] InputStream is = new SolrResourceLoader().openResource("solrj/sampleSimpleGroupResponse.xml");
 [ecj-lint]                 ^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 40. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/solrj/src/test/org/apache/solr/client/solrj/response/TestClusteringResponse.java (at line 41)
 [ecj-lint] InputStream is = new SolrResourceLoader().openResource("solrj/sampleClusteringResponse.xml");
 [ecj-lint]                 ^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 41. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/solrj/src/test/org/apache/solr/common/util/ContentStreamTest.java (at line 48)
 [ecj-lint] InputStream is = new SolrResourceLoader().openResource( "solrj/README" );
 [ecj-lint]                 ^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
 [ecj-lint] ----------
 [ecj-lint] 42. WARNING in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/solrj/src/test/org/apache/solr/common/util/ContentStreamTest.java (at line 77)
 [ecj-lint] InputStream is = new SolrResourceLoader().openResource( "solrj/README" );
 [ecj-lint]                 ^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
 [ecj-lint] ----------
 [ecj-lint] 42 problems (1 error, 41 warnings)

BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:836: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:101: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build.xml:684: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:2111: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:2144: Compile failed; see the compiler error output for details.

Total time: 105 minutes 26 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1617 - Still unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1617/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  org.apache.solr.cloud.MissingSegmentRecoveryTest.testLeaderRecovery

Error Message:
Expected a collection with one shard and two replicas null Live Nodes: [127.0.0.1:44343_solr, 127.0.0.1:58709_solr] Last available state: DocCollection(MissingSegmentRecoveryTest//collections/MissingSegmentRecoveryTest/state.json/8)={   "pullReplicas":"0",   "replicationFactor":"2",   "shards":{"shard1":{       "range":"80000000-7fffffff",       "state":"active",       "replicas":{         "core_node3":{           "core":"MissingSegmentRecoveryTest_shard1_replica_n1",           "base_url":"http://127.0.0.1:44343/solr",           "node_name":"127.0.0.1:44343_solr",           "state":"active",           "type":"NRT",           "leader":"true"},         "core_node4":{           "core":"MissingSegmentRecoveryTest_shard1_replica_n2",           "base_url":"http://127.0.0.1:58709/solr",           "node_name":"127.0.0.1:58709_solr",           "state":"down",           "type":"NRT"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"1",   "autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0"}

Stack T