[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 46 - Unstable!

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 46 - Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/46/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 8 in http://127.0.0.1:42438/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 8 in http://127.0.0.1:42438/solr
        at __randomizedtesting.SeedInfo.seed([79D3A88AD0E0ADFF:B823D126FDB06758]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNotNull(Assert.java:526)
        at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:868)
        at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:589)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 11499 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestTlogReplica
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_79D3A88AD0E0ADFF-001/init-core-data-001
   [junit4]   2> 778666 WARN  (SUITE-TestTlogReplica-seed#[79D3A88AD0E0ADFF]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 778666 INFO  (SUITE-TestTlogReplica-seed#[79D3A88AD0E0ADFF]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 778667 INFO  (SUITE-TestTlogReplica-seed#[79D3A88AD0E0ADFF]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 778668 INFO  (SUITE-TestTlogReplica-seed#[79D3A88AD0E0ADFF]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_79D3A88AD0E0ADFF-001/tempDir-001
   [junit4]   2> 778668 INFO  (SUITE-TestTlogReplica-seed#[79D3A88AD0E0ADFF]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 778668 INFO  (Thread-1867) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 778668 INFO  (Thread-1867) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 778669 ERROR (Thread-1867) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 778769 INFO  (SUITE-TestTlogReplica-seed#[79D3A88AD0E0ADFF]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:50764
   [junit4]   2> 778782 INFO  (jetty-launcher-1307-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 778782 INFO  (jetty-launcher-1307-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 778786 INFO  (jetty-launcher-1307-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6ad0517c{/solr,null,AVAILABLE}
   [junit4]   2> 778786 INFO  (jetty-launcher-1307-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4322bd3f{/solr,null,AVAILABLE}
   [junit4]   2> 778786 INFO  (jetty-launcher-1307-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@69dcd4b7{HTTP/1.1,[http/1.1]}{127.0.0.1:42438}
   [junit4]   2> 778786 INFO  (jetty-launcher-1307-thread-1) [    ] o.e.j.s.Server Started @781001ms
   [junit4]   2> 778786 INFO  (jetty-launcher-1307-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7892e48d{HTTP/1.1,[http/1.1]}{127.0.0.1:58018}
   [junit4]   2> 778786 INFO  (jetty-launcher-1307-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42438}
   [junit4]   2> 778786 INFO  (jetty-launcher-1307-thread-2) [    ] o.e.j.s.Server Started @781001ms
   [junit4]   2> 778786 INFO  (jetty-launcher-1307-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58018}
   [junit4]   2> 778787 ERROR (jetty-launcher-1307-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 778787 ERROR (jetty-launcher-1307-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 778787 INFO  (jetty-launcher-1307-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.1.0
   [junit4]   2> 778787 INFO  (jetty-launcher-1307-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.1.0
   [junit4]   2> 778787 INFO  (jetty-launcher-1307-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 778787 INFO  (jetty-launcher-1307-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 778787 INFO  (jetty-launcher-1307-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 778787 INFO  (jetty-launcher-1307-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-17T01:02:25.111Z
   [junit4]   2> 778787 INFO  (jetty-launcher-1307-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 778787 INFO  (jetty-launcher-1307-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-17T01:02:25.111Z
   [junit4]   2> 778794 INFO  (jetty-launcher-1307-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 778795 INFO  (jetty-launcher-1307-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 778807 INFO  (jetty-launcher-1307-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50764/solr
   [junit4]   2> 778812 INFO  (jetty-launcher-1307-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50764/solr
   [junit4]   2> 778956 INFO  (jetty-launcher-1307-thread-2) [n:127.0.0.1:58018_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 778956 INFO  (jetty-launcher-1307-thread-1) [n:127.0.0.1:42438_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 778957 INFO  (jetty-launcher-1307-thread-1) [n:127.0.0.1:42438_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42438_solr
   [junit4]   2> 778958 INFO  (jetty-launcher-1307-thread-2) [n:127.0.0.1:58018_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58018_solr
   [junit4]   2> 778959 INFO  (jetty-launcher-1307-thread-1) [n:127.0.0.1:42438_solr    ] o.a.s.c.Overseer Overseer (id=98320603217461254-127.0.0.1:42438_solr-n_0000000000) starting
   [junit4]   2> 778960 INFO  (zkCallback-1318-thread-1-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 778961 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 778969 INFO  (jetty-launcher-1307-thread-1) [n:127.0.0.1:42438_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42438_solr
   [junit4]   2> 778971 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 778972 INFO  (zkCallback-1318-thread-1-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 779022 INFO  (jetty-launcher-1307-thread-1) [n:127.0.0.1:42438_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42438.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@367cb8c9
   [junit4]   2> 779025 INFO  (jetty-launcher-1307-thread-1) [n:127.0.0.1:42438_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42438.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@367cb8c9
   [junit4]   2> 779025 INFO  (jetty-launcher-1307-thread-1) [n:127.0.0.1:42438_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42438.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@367cb8c9
   [junit4]   2> 779025 INFO  (jetty-launcher-1307-thread-1) [n:127.0.0.1:42438_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_79D3A88AD0E0ADFF-001/tempDir-001/node1/.
   [junit4]   2> 779062 INFO  (jetty-launcher-1307-thread-2) [n:127.0.0.1:58018_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58018.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@367cb8c9
   [junit4]   2> 779064 INFO  (jetty-launcher-1307-thread-2) [n:127.0.0.1:58018_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58018.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@367cb8c9
   [junit4]   2> 779064 INFO  (jetty-launcher-1307-thread-2) [n:127.0.0.1:58018_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58018.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@367cb8c9
   [junit4]   2> 779064 INFO  (jetty-launcher-1307-thread-2) [n:127.0.0.1:58018_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_79D3A88AD0E0ADFF-001/tempDir-001/node2/.
   [junit4]   2> 779083 INFO  (SUITE-TestTlogReplica-seed#[79D3A88AD0E0ADFF]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 779084 INFO  (SUITE-TestTlogReplica-seed#[79D3A88AD0E0ADFF]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:50764/solr ready
   [junit4]   2> 779088 INFO  (SUITE-TestTlogReplica-seed#[79D3A88AD0E0ADFF]-worker) [    ] o.a.s.c.TestTlogReplica Using legacyCloud?: false
   [junit4]   2> 779091 INFO  (qtp777095886-7808) [n:127.0.0.1:42438_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 779092 INFO  (qtp777095886-7808) [n:127.0.0.1:42438_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 779109 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testKillTlogReplica
   [junit4]   2> 779110 INFO  (qtp777095886-7809) [n:127.0.0.1:42438_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=0&collection.configName=conf&maxShardsPerNode=100&name=tlog_replica_test_kill_tlog_replica&nrtReplicas=0&action=CREATE&numShards=1&tlogReplicas=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 779113 INFO  (OverseerThreadFactory-3670-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.CreateCollectionCmd Create collection tlog_replica_test_kill_tlog_replica
   [junit4]   2> 779226 INFO  (OverseerStateUpdate-98320603217461254-127.0.0.1:42438_solr-n_0000000000) [n:127.0.0.1:42438_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"tlog_replica_test_kill_tlog_replica",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"tlog_replica_test_kill_tlog_replica_shard1_replica_t1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:58018/solr",
   [junit4]   2>   "type":"TLOG"}
   [junit4]   2> 779230 INFO  (OverseerStateUpdate-98320603217461254-127.0.0.1:42438_solr-n_0000000000) [n:127.0.0.1:42438_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"tlog_replica_test_kill_tlog_replica",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"tlog_replica_test_kill_tlog_replica_shard1_replica_t2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42438/solr",
   [junit4]   2>   "type":"TLOG"}
   [junit4]   2> 779434 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node1&collection.configName=conf&newCollection=true&name=tlog_replica_test_kill_tlog_replica_shard1_replica_t1&action=CREATE&numShards=1&collection=tlog_replica_test_kill_tlog_replica&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 779435 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=tlog_replica_test_kill_tlog_replica_shard1_replica_t2&action=CREATE&numShards=1&collection=tlog_replica_test_kill_tlog_replica&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 779435 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 779436 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 779552 INFO  (zkCallback-1318-thread-1-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 779552 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 779553 INFO  (zkCallback-1318-thread-2-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 779553 INFO  (zkCallback-1319-thread-2-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 780451 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 780451 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 780468 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.s.IndexSchema [tlog_replica_test_kill_tlog_replica_shard1_replica_t1] Schema name=minimal
   [junit4]   2> 780469 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.s.IndexSchema [tlog_replica_test_kill_tlog_replica_shard1_replica_t2] Schema name=minimal
   [junit4]   2> 780470 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 780470 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_kill_tlog_replica_shard1_replica_t1' using configuration from collection tlog_replica_test_kill_tlog_replica, trusted=true
   [junit4]   2> 780470 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 780470 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_kill_tlog_replica_shard1_replica_t2' using configuration from collection tlog_replica_test_kill_tlog_replica, trusted=true
   [junit4]   2> 780470 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58018.solr.core.tlog_replica_test_kill_tlog_replica.shard1.replica_t1' (registry 'solr.core.tlog_replica_test_kill_tlog_replica.shard1.replica_t1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@367cb8c9
   [junit4]   2> 780471 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 780471 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42438.solr.core.tlog_replica_test_kill_tlog_replica.shard1.replica_t2' (registry 'solr.core.tlog_replica_test_kill_tlog_replica.shard1.replica_t2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@367cb8c9
   [junit4]   2> 780471 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.SolrCore [[tlog_replica_test_kill_tlog_replica_shard1_replica_t1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_79D3A88AD0E0ADFF-001/tempDir-001/node2/tlog_replica_test_kill_tlog_replica_shard1_replica_t1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_79D3A88AD0E0ADFF-001/tempDir-001/node2/./tlog_replica_test_kill_tlog_replica_shard1_replica_t1/data/]
   [junit4]   2> 780471 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 780471 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.SolrCore [[tlog_replica_test_kill_tlog_replica_shard1_replica_t2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_79D3A88AD0E0ADFF-001/tempDir-001/node1/tlog_replica_test_kill_tlog_replica_shard1_replica_t2], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_79D3A88AD0E0ADFF-001/tempDir-001/node1/./tlog_replica_test_kill_tlog_replica_shard1_replica_t2/data/]
   [junit4]   2> 780527 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 780527 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 780528 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 780528 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 780530 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4748cd31[tlog_replica_test_kill_tlog_replica_shard1_replica_t1] main]
   [junit4]   2> 780532 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 780532 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 780532 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 780533 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 780533 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 780533 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 780533 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 780534 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1573129653434974208
   [junit4]   2> 780535 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.s.SolrIndexSearcher Opening [Searcher@6d47dba4[tlog_replica_test_kill_tlog_replica_shard1_replica_t2] main]
   [junit4]   2> 780535 INFO  (searcherExecutor-3675-thread-1-processing-n:127.0.0.1:58018_solr x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1 s:shard1 c:tlog_replica_test_kill_tlog_replica r:core_node1) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.SolrCore [tlog_replica_test_kill_tlog_replica_shard1_replica_t1] Registered new searcher Searcher@4748cd31[tlog_replica_test_kill_tlog_replica_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 780536 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 780536 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 780537 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 780538 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1573129653439168512
   [junit4]   2> 780539 INFO  (searcherExecutor-3676-thread-1-processing-n:127.0.0.1:42438_solr x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2 s:shard1 c:tlog_replica_test_kill_tlog_replica r:core_node2) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.SolrCore [tlog_replica_test_kill_tlog_replica_shard1_replica_t2] Registered new searcher Searcher@6d47dba4[tlog_replica_test_kill_tlog_replica_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 780543 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 780543 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 780543 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:58018/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t1/
   [junit4]   2> 780544 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_kill_tlog_replica_shard1_replica_t1 url=http://127.0.0.1:58018/solr START replicas=[http://127.0.0.1:42438/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t2/] nUpdates=100
   [junit4]   2> 780544 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_kill_tlog_replica_shard1_replica_t1 url=http://127.0.0.1:58018/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 780546 INFO  (qtp777095886-7813) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 780546 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 780546 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 780546 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 780547 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.ZkController tlog_replica_test_kill_tlog_replica_shard1_replica_t1 stopping background replication from leader
   [junit4]   2> 780551 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:58018/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t1/ shard1
   [junit4]   2> 780655 INFO  (zkCallback-1319-thread-2-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 780655 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 780655 INFO  (zkCallback-1318-thread-2-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 780655 INFO  (zkCallback-1318-thread-1-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 780704 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 780706 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node1&collection.configName=conf&newCollection=true&name=tlog_replica_test_kill_tlog_replica_shard1_replica_t1&action=CREATE&numShards=1&collection=tlog_replica_test_kill_tlog_replica&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1272
   [junit4]   2> 780809 INFO  (zkCallback-1319-thread-2-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 780809 INFO  (zkCallback-1318-thread-2-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 780809 INFO  (zkCallback-1318-thread-1-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 780809 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 781547 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.ZkController tlog_replica_test_kill_tlog_replica_shard1_replica_t2 starting background replication from leader
   [junit4]   2> 781547 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
   [junit4]   2> 781553 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
   [junit4]   2> 781557 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=tlog_replica_test_kill_tlog_replica_shard1_replica_t2&action=CREATE&numShards=1&collection=tlog_replica_test_kill_tlog_replica&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=2122
   [junit4]   2> 781564 INFO  (qtp777095886-7809) [n:127.0.0.1:42438_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 781623 INFO  (indexFetcher-3686-thread-1) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.IndexFetcher Updated masterUrl to http://127.0.0.1:58018/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t1/
   [junit4]   2> 781626 INFO  (qtp896400729-7819) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 781643 INFO  (indexFetcher-3686-thread-1) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 781643 INFO  (indexFetcher-3686-thread-1) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 781643 INFO  (indexFetcher-3686-thread-1) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 781643 INFO  (indexFetcher-3686-thread-1) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 781664 INFO  (zkCallback-1318-thread-1-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 781664 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 781664 INFO  (zkCallback-1319-thread-2-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 781664 INFO  (zkCallback-1318-thread-2-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 782565 INFO  (qtp777095886-7809) [n:127.0.0.1:42438_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=0&collection.configName=conf&maxShardsPerNode=100&name=tlog_replica_test_kill_tlog_replica&nrtReplicas=0&action=CREATE&numShards=1&tlogReplicas=2&wt=javabin&version=2} status=0 QTime=3456
   [junit4]   2> 782570 INFO  (qtp896400729-7821) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 782573 INFO  (qtp777095886-7806) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 782577 INFO  (qtp777095886-7808) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:58018/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t1/&wt=javabin&version=2}{add=[1 (1573129655574069248)]} 0 0
   [junit4]   2> 782577 INFO  (qtp896400729-7822) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1 (1573129655574069248)]} 0 3
   [junit4]   2> 782579 INFO  (qtp777095886-7813) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:58018/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
   [junit4]   2> 782579 INFO  (qtp896400729-7819) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1573129655579312128,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 782579 INFO  (qtp896400729-7819) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@79f0987 commitCommandVersion:1573129655579312128
   [junit4]   2> 782582 INFO  (qtp896400729-7819) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2471a02d[tlog_replica_test_kill_tlog_replica_shard1_replica_t1] main]
   [junit4]   2> 782583 INFO  (qtp896400729-7819) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 782584 INFO  (searcherExecutor-3675-thread-1-processing-n:127.0.0.1:58018_solr x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1 s:shard1 c:tlog_replica_test_kill_tlog_replica r:core_node1) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.SolrCore [tlog_replica_test_kill_tlog_replica_shard1_replica_t1] Registered new searcher Searcher@2471a02d[tlog_replica_test_kill_tlog_replica_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):C1)))}
   [junit4]   2> 782584 INFO  (qtp896400729-7819) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:58018/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 4
   [junit4]   2> 782584 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/update params={_stateVer_=tlog_replica_test_kill_tlog_replica:5&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 6
   [junit4]   2> 782586 INFO  (qtp896400729-7820) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
   [junit4]   2> 782588 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 782689 INFO  (qtp777095886-7809) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 782791 INFO  (qtp777095886-7814) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 782892 INFO  (qtp777095886-7814) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 782993 INFO  (qtp777095886-7812) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 783094 INFO  (qtp777095886-7808) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 783120 INFO  (OverseerCollectionConfigSetProcessor-98320603217461254-127.0.0.1:42438_solr-n_0000000000) [n:127.0.0.1:42438_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 783195 INFO  (qtp777095886-7813) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 783296 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 783397 INFO  (qtp777095886-7809) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 783498 INFO  (qtp777095886-7806) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 783600 INFO  (qtp777095886-7814) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 783701 INFO  (qtp777095886-7812) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 783806 INFO  (qtp777095886-7808) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 783907 INFO  (qtp777095886-7813) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 784009 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 784110 INFO  (qtp777095886-7809) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 784211 INFO  (qtp777095886-7806) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 784313 INFO  (qtp777095886-7814) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 784416 INFO  (qtp777095886-7812) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 784517 INFO  (qtp777095886-7812) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 784618 INFO  (qtp777095886-7813) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 784624 INFO  (qtp896400729-7821) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 784625 INFO  (indexFetcher-3686-thread-1) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.IndexFetcher Master's generation: 2
   [junit4]   2> 784625 INFO  (indexFetcher-3686-thread-1) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.IndexFetcher Master's version: 1500253348903
   [junit4]   2> 784625 INFO  (indexFetcher-3686-thread-1) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 784625 INFO  (indexFetcher-3686-thread-1) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 784625 INFO  (indexFetcher-3686-thread-1) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.IndexFetcher Starting replication process
   [junit4]   2> 784625 INFO  (qtp896400729-7822) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=0
   [junit4]   2> 784626 INFO  (indexFetcher-3686-thread-1) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.IndexFetcher Number of files in latest index in master: 19
   [junit4]   2> 784626 INFO  (indexFetcher-3686-thread-1) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(RAMDirectory@581cf514 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1edb38d0)
   [junit4]   2> 784626 INFO  (qtp896400729-7818) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784627 INFO  (qtp896400729-7819) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene50_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784628 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene50_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784629 INFO  (qtp896400729-7820) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784630 INFO  (qtp896400729-7811) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784630 INFO  (qtp896400729-7821) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784631 INFO  (qtp896400729-7822) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene50_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784632 INFO  (qtp896400729-7818) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784633 INFO  (qtp896400729-7819) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784633 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene50_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784634 INFO  (qtp896400729-7820) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784634 INFO  (qtp896400729-7811) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene70_0.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784635 INFO  (qtp896400729-7821) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784636 INFO  (qtp896400729-7822) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene70_0.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784636 INFO  (qtp896400729-7818) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784637 INFO  (qtp896400729-7819) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784638 INFO  (qtp896400729-7817) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784639 INFO  (qtp896400729-7820) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784639 INFO  (qtp896400729-7811) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=segments_2&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 784640 INFO  (indexFetcher-3686-thread-1) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=3030) : 0 secs (null bytes/sec) to MockDirectoryWrapper(RAMDirectory@581cf514 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1edb38d0)
   [junit4]   2> 784642 INFO  (indexFetcher-3686-thread-1) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 784643 INFO  (indexFetcher-3686-thread-1) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.s.SolrIndexSearcher Opening [Searcher@17763987[tlog_replica_test_kill_tlog_replica_shard1_replica_t2] main]
   [junit4]   2> 784643 INFO  (searcherExecutor-3676-thread-1-processing-n:127.0.0.1:42438_solr x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2 s:shard1 c:tlog_replica_test_kill_tlog_replica r:core_node2) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.SolrCore [tlog_replica_test_kill_tlog_replica_shard1_replica_t2] Registered new searcher Searcher@17763987[tlog_replica_test_kill_tlog_replica_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):C1)))}
   [junit4]   2> 784719 INFO  (qtp777095886-7810) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
   [junit4]   2> 784720 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.c.ChaosMonkey monkey: kill jetty! 58018
   [junit4]   2> 784722 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@7892e48d{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 784723 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1286326702
   [junit4]   2> 784724 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 784727 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 784730 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 784733 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 784736 INFO  (coreCloseExecutor-3688-thread-1) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.SolrCore [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  CLOSING SolrCore org.apache.solr.core.SolrCore@3d55cbd2
   [junit4]   2> 784739 INFO  (coreCloseExecutor-3688-thread-1) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.tlog_replica_test_kill_tlog_replica.shard1.replica_t1, tag=1029032914
   [junit4]   2> 784742 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 784742 INFO  (zkCallback-1319-thread-2-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 784743 INFO  (zkCallback-1325-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 784744 INFO  (zkCallback-1318-thread-2-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 784745 INFO  (zkCallback-1318-thread-1-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 784746 INFO  (coreCloseExecutor-3688-thread-1) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.tlog_replica_test_kill_tlog_replica.shard1.leader, tag=1029032914
   [junit4]   2> 784750 INFO  (zkCallback-1325-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 784751 INFO  (zkCallback-1319-thread-2-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 784755 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 784860 INFO  (zkCallback-1319-thread-2-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 784860 INFO  (zkCallback-1325-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 784860 INFO  (zkCallback-1319-thread-3-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 786250 WARN  (zkCallback-1318-thread-1-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 786251 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4322bd3f{/solr,null,UNAVAILABLE}
   [junit4]   2> 786253 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.c.ChaosMonkey monkey: stop jetty! 58018
   [junit4]   2> 787258 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42438/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t2/
   [junit4]   2> 787258 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 787258 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.SyncStrategy http://127.0.0.1:42438/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t2/ has no replicas
   [junit4]   2> 787259 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.ZkController tlog_replica_test_kill_tlog_replica_shard1_replica_t2 stopping background replication from leader
   [junit4]   2> 787260 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext New leader does not have old tlog to replay
   [junit4]   2> 787266 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42438/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t2/ shard1
   [junit4]   2> 787272 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 787272 INFO  (zkCallback-1319-thread-2-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 787272 INFO  (zkCallback-1325-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 787324 INFO  (qtp777095886-7809) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2 (1573129660552708096)]} 0 1065
   [junit4]   2> 787325 INFO  (qtp777095886-7808) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1573129660555853824,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 787325 INFO  (qtp777095886-7808) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@40768b4 commitCommandVersion:1573129660555853824
   [junit4]   2> 787329 INFO  (qtp777095886-7808) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.s.SolrIndexSearcher Opening [Searcher@e25e0e0[tlog_replica_test_kill_tlog_replica_shard1_replica_t2] main]
   [junit4]   2> 787330 INFO  (qtp777095886-7808) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 787331 INFO  (searcherExecutor-3676-thread-1-processing-n:127.0.0.1:42438_solr x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2 s:shard1 c:tlog_replica_test_kill_tlog_replica r:core_node2) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.SolrCore [tlog_replica_test_kill_tlog_replica_shard1_replica_t2] Registered new searcher Searcher@e25e0e0[tlog_replica_test_kill_tlog_replica_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):C1) Uninverting(_1(7.1.0):C1)))}
   [junit4]   2> 787331 INFO  (qtp777095886-7808) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/update params={_stateVer_=tlog_replica_test_kill_tlog_replica:8&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 6
   [junit4]   2> 787335 INFO  (qtp777095886-7812) [n:127.0.0.1:42438_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node2 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=2 status=0 QTime=0
   [junit4]   2> 787336 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.c.ChaosMonkey monkey: starting jetty! 58018
   [junit4]   2> 787337 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 787339 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@d14c333{/solr,null,AVAILABLE}
   [junit4]   2> 787339 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@e0fd735{HTTP/1.1,[http/1.1]}{127.0.0.1:58018}
   [junit4]   2> 787339 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.e.j.s.Server Started @789556ms
   [junit4]   2> 787339 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58018}
   [junit4]   2> 787341 ERROR (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 787341 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.1.0
   [junit4]   2> 787341 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 787341 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 787342 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-17T01:02:33.666Z
   [junit4]   2> 787351 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 787368 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50764/solr
   [junit4]   2> 787386 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [n:127.0.0.1:58018_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 787389 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [n:127.0.0.1:58018_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 787391 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [n:127.0.0.1:58018_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58018_solr
   [junit4]   2> 787394 INFO  (zkCallback-1319-thread-2-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 787395 INFO  (zkCallback-1330-thread-1-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 787395 INFO  (zkCallback-1325-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 787792 INFO  (zkCallback-1319-thread-2-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 787793 INFO  (zkCallback-1325-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 787793 INFO  (zkCallback-1319-thread-1-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 787797 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [n:127.0.0.1:58018_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58018.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@367cb8c9
   [junit4]   2> 787799 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [n:127.0.0.1:58018_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58018.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@367cb8c9
   [junit4]   2> 787800 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [n:127.0.0.1:58018_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58018.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@367cb8c9
   [junit4]   2> 787801 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [n:127.0.0.1:58018_solr    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_79D3A88AD0E0ADFF-001/tempDir-001/node2/.
   [junit4]   2> 787801 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[79D3A88AD0E0ADFF]) [n:127.0.0.1:58018_solr    ] o.a.s.c.CorePropertiesLocator Cores are: [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]
   [junit4]   2> 787803 INFO  (coreLoadExecutor-3695-thread-1-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 787812 INFO  (coreLoadExecutor-3695-thread-1-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 787817 INFO  (coreLoadExecutor-3695-thread-1-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.s.IndexSchema [tlog_replica_test_kill_tlog_replica_shard1_replica_t1] Schema name=minimal
   [junit4]   2> 787819 INFO  (coreLoadExecutor-3695-thread-1-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 787819 INFO  (coreLoadExecutor-3695-thread-1-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_kill_tlog_replica_shard1_replica_t1' using configuration from collection tlog_replica_test_kill_tlog_replica, trusted=true
   [junit4]   2> 787819 INFO  (coreLoadExecutor-3695-thread-1-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node1 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58018.solr.core.tlog_replica_test_kill_tlog_replica.shard1.replica_t1' (registry 'solr.core.tlog_replica_test_kill_tlog_replica.shard1.replica_t1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@367cb8c9
   [junit4]   2> 787820 INFO  (coreLoadExecutor-3695-thread-1-processing-n:127.0.0.1:58018_solr) [n:127.0.0

[...truncated too long message...]

885761 INFO  (jetty-closer-1308-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 885761 INFO  (jetty-closer-1308-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 885766 INFO  (jetty-closer-1308-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 885766 INFO  (jetty-closer-1308-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 885770 INFO  (jetty-closer-1308-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 885770 INFO  (jetty-closer-1308-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 885771 INFO  (jetty-closer-1308-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 885773 INFO  (jetty-closer-1308-thread-2) [    ] o.a.s.c.Overseer Overseer (id=98320603217461265-127.0.0.1:58018_solr-n_0000000004) closing
   [junit4]   2> 885773 INFO  (jetty-closer-1308-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 885773 INFO  (OverseerStateUpdate-98320603217461265-127.0.0.1:58018_solr-n_0000000004) [n:127.0.0.1:58018_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:58018_solr
   [junit4]   2> 885778 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15d4e1154e10011, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 887290 WARN  (zkCallback-1345-thread-3-processing-n:127.0.0.1:58018_solr) [n:127.0.0.1:58018_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 887293 WARN  (zkCallback-1376-thread-2-processing-n:127.0.0.1:42438_solr) [n:127.0.0.1:42438_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 887295 INFO  (jetty-closer-1308-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5f4683c0{/solr,null,UNAVAILABLE}
   [junit4]   2> 887301 INFO  (jetty-closer-1308-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2ac01cda{/solr,null,UNAVAILABLE}
   [junit4]   2> 887302 ERROR (SUITE-TestTlogReplica-seed#[79D3A88AD0E0ADFF]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 887305 INFO  (SUITE-TestTlogReplica-seed#[79D3A88AD0E0ADFF]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:50764 50764
   [junit4]   2> 892730 INFO  (Thread-1867) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:50764 50764
   [junit4]   2> 892734 WARN  (Thread-1867) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 32 /solr/configs/conf
   [junit4]   2> 10 /solr/aliases.json
   [junit4]   2> 9 /solr/security.json
   [junit4]   2> 3 /solr/clusterprops.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 36 /solr/collections/tlog_replica_test_recovery/state.json
   [junit4]   2> 28 /solr/collections/tlog_replica_test_create_delete/state.json
   [junit4]   2> 27 /solr/collections/tlog_replica_test_remove_leader/state.json
   [junit4]   2> 26 /solr/collections/tlog_replica_test_kill_leader/state.json
   [junit4]   2> 25 /solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/state.json
   [junit4]   2> 24 /solr/collections/tlog_replica_test_kill_tlog_replica/state.json
   [junit4]   2> 23 /solr/collections/tlog_replica_test_add_remove_tlog_replica/state.json
   [junit4]   2> 21 /solr/collections/tlog_replica_test_basic_leader_election/state.json
   [junit4]   2> 15 /solr/collections/tlog_replica_test_only_leader_indexes/state.json
   [junit4]   2> 13 /solr/collections/tlog_replica_test_delete_by_id/state.json
   [junit4]   2> 10 /solr/clusterprops.json
   [junit4]   2> 10 /solr/clusterstate.json
   [junit4]   2> 6 /solr/collections/tlog_replica_test_add_docs/state.json
   [junit4]   2> 6 /solr/collections/tlog_replica_test_real_time_get/state.json
   [junit4]   2> 4 /solr/overseer_elect/election/98320603217461265-127.0.0.1:58018_solr-n_0000000004
   [junit4]   2> 4 /solr/collections/tlog_replica_test_recovery/leader_elect/shard1/election/98320603217461265-core_node2-n_0000000000
   [junit4]   2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/98320603217461259-core_node3-n_0000000000
   [junit4]   2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard2/election/98320603217461259-core_node5-n_0000000000
   [junit4]   2> 2 /solr/overseer_elect/election/98320603217461254-127.0.0.1:42438_solr-n_0000000000
   [junit4]   2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard2/election/98320603217461259-core_node7-n_0000000000
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 10 /solr/collections
   [junit4]   2> 4 /solr/overseer/queue
   [junit4]   2> 4 /solr/overseer/collection-queue-work
   [junit4]   2> 3 /solr/live_nodes
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_79D3A88AD0E0ADFF-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {foo=Lucene50(blocksize=128), title_s=PostingsFormat(name=LuceneVarGapFixedInterval), id=PostingsFormat(name=Direct), foo_s=Lucene50(blocksize=128)}, docValues:{_version_=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Asserting), inplace_updatable_int=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=618, maxMBSortInHeap=7.578449408898537, sim=RandomSimilarity(queryNorm=true): {}, locale=bg, timezone=Africa/Niamey
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=3,threads=1,free=145791184,total=314191872
   [junit4]   2> NOTE: All tests run in this JVM: [DistributedFacetPivotLargeTest, TestDFISimilarityFactory, ActionThrottleTest, NodeMutatorTest, TestChildDocTransformer, TestMaxTokenLenTokenizer, TestSerializedLuceneMatchVersion, TestCrossCoreJoin, DocValuesNotIndexedTest, TestRequestForwarding, SpellingQueryConverterTest, TestJsonRequest, TestLegacyFieldReuse, DistributedTermsComponentTest, TestFreeTextSuggestions, TestJoin, TestSolrCloudWithDelegationTokens, V2ApiIntegrationTest, TestFiltering, PingRequestHandlerTest, TestMergePolicyConfig, TestSchemaSimilarityResource, TestDocBasedVersionConstraints, ChaosMonkeySafeLeaderTest, SuggestComponentTest, DirectSolrSpellCheckerTest, HdfsBasicDistributedZk2Test, TestBM25SimilarityFactory, TestSolrIndexConfig, EchoParamsTest, HdfsRestartWhileUpdatingTest, TestStressInPlaceUpdates, TestLeaderElectionWithEmptyReplica, CoreMergeIndexesAdminHandlerTest, ZkSolrClientTest, CloudExitableDirectoryReaderTest, TestNumericRangeQuery32, ConnectionManagerTest, SpellPossibilityIteratorTest, FileUtilsTest, TestFieldCacheVsDocValues, AssignTest, TestSolrCloudWithSecureImpersonation, DistributedQueueTest, TestReloadAndDeleteDocs, TestBulkSchemaConcurrent, TestRuleBasedAuthorizationPlugin, HdfsCollectionsAPIDistributedZkTest, TestTrie, TestDistributedMissingSort, TestSystemIdResolver, SolrMetricReporterTest, TestRandomFlRTGCloud, SynonymTokenizerTest, DocumentBuilderTest, TestExactSharedStatsCache, TestManagedSchemaThreadSafety, MoreLikeThisHandlerTest, TestManagedSynonymGraphFilterFactory, MBeansHandlerTest, TestExpandComponent, TestNumericTerms64, CurrencyFieldTypeTest, CollectionStateFormat2Test, SearchHandlerTest, SearchHandlerTest, TestSolrCloudWithHadoopAuthPlugin, DistributedQueryElevationComponentTest, RollingRestartTest, CoreAdminCreateDiscoverTest, SecurityConfHandlerTest, DistributedFacetPivotWhiteBoxTest, TestRangeQuery, TermVectorComponentDistributedTest, TestSolrDeletionPolicy1, TestTestInjection, TestCloudPivotFacet, TestFastOutputStream, ParsingFieldUpdateProcessorsTest, TestAnalyzedSuggestions, PreAnalyzedFieldTest, TestSchemaNameResource, RulesTest, TestSortByMinMaxFunction, TestWordDelimiterFilterFactory, TestConfigSetsAPIExclusivity, TestNamedUpdateProcessors, TestIntervalFaceting, TestGraphMLResponseWriter, TestPullReplica, TestConfigReload, BasicAuthStandaloneTest, TestSearchPerf, TestXmlQParser, TestRestManager, TestShortCircuitedRequests, TestInPlaceUpdatesStandalone, TestUnifiedSolrHighlighter, CursorMarkTest, TestPerFieldSimilarity, BlockJoinFacetDistribTest, AnalyticsMergeStrategyTest, ConvertedLegacyTest, TestConfigSets, TestConfigSetImmutable, ConcurrentDeleteAndCreateCollectionTest, TriLevelCompositeIdRoutingTest, CollectionsAPIAsyncDistributedZkTest, SimpleMLTQParserTest, TestDelegationWithHadoopAuth, DeleteStatusTest, BadComponentTest, TestPerFieldSimilarityWithDefaultOverride, SharedFSAutoReplicaFailoverTest, TestPayloadScoreQParserPlugin, TestManagedResource, JavabinLoaderTest, TestFieldCollectionResource, BufferStoreTest, TestTlogReplica]
   [junit4] Completed [203/728 (1!)] on J1 in 114.08s, 13 tests, 1 failure <<< FAILURES!

[...truncated 45043 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 47 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/47/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.core.TestJmxIntegration.testJmxRegistration

Error Message:
org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed

Stack Trace:
javax.management.RuntimeMBeanException: org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed
        at __randomizedtesting.SeedInfo.seed([F830FF2FC67C0210:76E19B15AB3D5A75]:0)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
        at org.apache.solr.core.TestJmxIntegration.testJmxRegistration(TestJmxIntegration.java:121)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed
        at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:268)
        at org.apache.lucene.index.StandardDirectoryReader.getVersion(StandardDirectoryReader.java:338)
        at org.apache.lucene.index.FilterDirectoryReader.getVersion(FilterDirectoryReader.java:119)
        at org.apache.lucene.index.FilterDirectoryReader.getVersion(FilterDirectoryReader.java:119)
        at org.apache.solr.search.SolrIndexSearcher.lambda$initializeMetrics$14(SolrIndexSearcher.java:2254)
        at com.codahale.metrics.JmxReporter$JmxGauge.getValue(JmxReporter.java:188)
        at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
        at sun.reflect.GeneratedMethodAccessor38.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
        at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
        at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
        at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
        at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
        at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
        ... 41 more




Build Log:
[...truncated 12184 lines...]
   [junit4] Suite: org.apache.solr.core.TestJmxIntegration
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestJmxIntegration_F830FF2FC67C0210-001/init-core-data-001
   [junit4]   2> 1555764 WARN  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=3 numCloses=3
   [junit4]   2> 1555764 INFO  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1555766 INFO  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 1555767 INFO  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 1555767 INFO  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/lib, /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1555796 INFO  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 1555808 INFO  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 1555863 INFO  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1555890 INFO  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f9adbc9
   [junit4]   2> 1555892 INFO  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f9adbc9
   [junit4]   2> 1555892 INFO  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f9adbc9
   [junit4]   2> 1555893 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/lib, /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1555911 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 1555923 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1555973 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1555979 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 1555980 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f9adbc9
   [junit4]   2> 1555980 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1555980 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestJmxIntegration_F830FF2FC67C0210-001/init-core-data-001/]
   [junit4]   2> 1555981 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=50, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6881435660185261]
   [junit4]   2> 1556021 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1556021 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1556021 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1556021 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1556022 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=12, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.637600252113695]
   [junit4]   2> 1556022 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4afa15db[collection1] main]
   [junit4]   2> 1556029 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1556032 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1556045 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1556045 INFO  (coreLoadExecutor-8157-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1573157578890805248
   [junit4]   2> 1556045 INFO  (searcherExecutor-8158-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1556046 INFO  (searcherExecutor-8158-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 1556046 INFO  (searcherExecutor-8158-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1556046 INFO  (searcherExecutor-8158-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1556046 INFO  (searcherExecutor-8158-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1556046 INFO  (searcherExecutor-8158-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 1556046 INFO  (searcherExecutor-8158-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 1556047 INFO  (searcherExecutor-8158-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 1556047 INFO  (searcherExecutor-8158-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 1556047 INFO  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 1556047 INFO  (searcherExecutor-8158-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4afa15db[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1556049 INFO  (TEST-TestJmxIntegration.testJmxRegistration-seed#[F830FF2FC67C0210]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testJmxRegistration
   [junit4]   2> 1556092 INFO  (TEST-TestJmxIntegration.testJmxRegistration-seed#[F830FF2FC67C0210]) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1556102 INFO  (TEST-TestJmxIntegration.testJmxRegistration-seed#[F830FF2FC67C0210]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testJmxRegistration
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestJmxIntegration -Dtests.method=testJmxRegistration -Dtests.seed=F830FF2FC67C0210 -Dtests.slow=true -Dtests.locale=vi-VN -Dtests.timezone=America/Kralendijk -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.06s J1 | TestJmxIntegration.testJmxRegistration <<<
   [junit4]    > Throwable #1: javax.management.RuntimeMBeanException: org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed
   [junit4]    > at __randomizedtesting.SeedInfo.seed([F830FF2FC67C0210:76E19B15AB3D5A75]:0)
   [junit4]    > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
   [junit4]    > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
   [junit4]    > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
   [junit4]    > at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
   [junit4]    > at org.apache.solr.core.TestJmxIntegration.testJmxRegistration(TestJmxIntegration.java:121)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed
   [junit4]    > at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:268)
   [junit4]    > at org.apache.lucene.index.StandardDirectoryReader.getVersion(StandardDirectoryReader.java:338)
   [junit4]    > at org.apache.lucene.index.FilterDirectoryReader.getVersion(FilterDirectoryReader.java:119)
   [junit4]    > at org.apache.lucene.index.FilterDirectoryReader.getVersion(FilterDirectoryReader.java:119)
   [junit4]    > at org.apache.solr.search.SolrIndexSearcher.lambda$initializeMetrics$14(SolrIndexSearcher.java:2254)
   [junit4]    > at com.codahale.metrics.JmxReporter$JmxGauge.getValue(JmxReporter.java:188)
   [junit4]    > at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
   [junit4]    > at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
   [junit4]    > at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
   [junit4]    > at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
   [junit4]    > at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
   [junit4]    > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
   [junit4]    > ... 41 more
   [junit4]   2> 1556106 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[F830FF2FC67C0210]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testJmxUpdate
   [junit4]   2> 1556113 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[F830FF2FC67C0210]) [    ] o.a.s.c.TestJmxIntegration Mbeans in server: [solr:dom1=core,dom2=collection1,category=QUERY,scope=/terms,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/dismax,name=handlerStart, java.lang:type=Threading, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mock,name=requests, solr:dom1=node,category=QUERY,scope=/admin/metrics/collector,name=handlerStart, solr:dom1=node,category=ADMIN,scope=/admin/metrics,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mock,name=errors, solr:dom1=jvm,name0=os,name=systemLoadAverage, solr:dom1=core,dom2=collection1,category=QUERY,scope=/get,name=errors, solr:dom1=node,category=QUERY,scope=/admin/autoscaling,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/defaults,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH1,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=requestTimes, solr:dom1=core,dom2=reloaded_collection,dom3=shard2,dom4=replica_n1,category=SEARCHER,scope=searcher,name=indexVersion, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=requestTimes, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=timeouts, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.updateExecutor.running, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mock,name=handlerStart, solr:dom1=node,category=QUERY,scope=httpShardHandler,name=threadPool.httpShardExecutor.duration, solr:dom1=node,category=ADMIN,scope=/admin/metrics,name=serverErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=handlerStart, solr:dom1=jvm,name0=buffers,name1=mapped,name=TotalCapacity, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/luke,name=requests, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=rollbacks, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=timeouts, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.recoveryExecutor.submitted, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/stream,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/dismax,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/terms,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/graph,name=clientErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json,name=timeouts, solr:dom1=node,category=CONTAINER,scope=threadPool,name=coreContainerWorkExecutor.running, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=requestTimes, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=reader, solr:dom1=jvm,name0=gc,name1=ConcurrentMarkSweep,name=count, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragmenter,name=gap.requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=requestTimes, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrBoundaryScanner,name=breakIterator.requests, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=requestTimes, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=handlerStart, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=errors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/stream,name=timeouts, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=errors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/file,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/sql,name=requests, solr:dom1=core,dom2=collection1,category=TLOG,name=state, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=errors, solr:dom1=jvm,name0=threads,name1=waiting,name=count, solr:dom1=jvm,name0=classes,name=unloaded, solr:dom1=jvm,name0=buffers,name1=direct,name=MemoryUsed, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=threadPool.parallelCoreAdminExecutor.submitted, solr:dom1=core,dom2=collection1,category=TLOG,scope=replay,name=remaining.bytes, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/properties,name=timeouts, solr:dom1=node,category=CONTAINER,scope=cores,name=loaded, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=handlerStart, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=handlerStart, solr:dom1=jvm,name0=memory,name1=pools,name2=Par-Survivor-Space,name=committed, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mltrh,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/sql,name=timeouts, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=new,name=errors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/config,name=clientErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=handlerStart, solr:dom1=node,category=QUERY,scope=httpShardHandler,name=threadPool.httpShardExecutor.completed, solr:dom1=core,dom2=collection1,category=QUERY,scope=/debug/dump,name=clientErrors, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/graph,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak_Direct,name=serverErrors, java.lang:type=Runtime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=cumulativeDeletesById, solr:dom1=jvm,name0=memory,name1=pools,name2=Par-Eden-Space,name=usage, solr:dom1=core,dom2=collection1,category=QUERY,scope=/defaults,name=timeouts, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=cumulativeDeletesByQuery, solr:dom1=core,dom2=collection1,category=QUERY,scope=/dismax,name=timeouts, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=errors, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.recoveryExecutor.duration, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak_Direct,name=timeouts, solr:dom1=core,dom2=collection1,category=CORE,scope=fs,name=totalSpace, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH_Direct,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/terms,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/mbeans,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=handlerStart, solr:dom1=node,category=CONTAINER,scope=threadPool,name=coreContainerWorkExecutor.duration, solr:dom1=node,category=ADMIN,scope=/admin/zookeeper,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak_Direct,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak_Direct,name=clientErrors, solr:dom1=node,category=CONTAINER,scope=cores,name=unloaded, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=serverErrors, solr:dom1=node,category=ADMIN,scope=/admin/metrics,name=requestTimes, solr:dom1=node,category=ADMIN,scope=/admin/metrics,name=timeouts, solr:dom1=core,dom2=collection1,category=CORE,scope=fs,name=usableSpace, solr:dom1=jvm,name0=os,name=totalPhysicalMemorySize, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH,name=serverErrors, solr:dom1=jvm,name0=os,name=totalSwapSpaceSize, solr:dom1=jvm,name0=memory,name1=pools,name2=Code-Cache,name=init, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=cumulativeErrors, solr:dom1=node,category=QUERY,scope=/admin/metrics/collector,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mlt,name=handlerStart, solr:dom1=jvm,name0=os,name=availableProcessors, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.recoveryExecutor.running, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak,name=timeouts, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak,name=errors, solr:dom1=node,category=ADMIN,scope=/admin/zookeeper,name=timeouts, solr:dom1=jvm,name0=threads,name=count, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=timeouts, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=caching, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mock,name=serverErrors, solr:dom1=jvm,name0=memory,name1=pools,name2=Code-Cache,name=usage, solr:dom1=jvm,name0=memory,name1=total,name=used, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFormatter,name=html.requests, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=clientErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=deletesByQuery, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=timeouts, solr:dom1=node,category=QUERY,scope=/admin/autoscaling,name=serverErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json/docs,name=serverErrors, solr:dom1=node,category=QUERY,scope=/admin/autoscaling,name=timeouts, solr:dom1=jvm,name0=threads,name1=new,name=count, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak,name=handlerStart, solr:dom1=core,dom2=collection1,category=TLOG,scope=replay,name=ops, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH1,name=handlerStart, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/logging,name=serverErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=requests, solr:dom1=jvm,name0=gc,name1=ParNew,name=time, solr:dom1=core,dom2=collection1,category=QUERY,scope=/stream,name=clientErrors, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=openedAt, solr:dom1=node,category=ADMIN,scope=/admin/configs,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mlt,name=requestTimes, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mock,name=initCount, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=clientErrors, java.lang:type=MemoryPool,name=Par Survivor Space, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak,name=requests, solr:dom1=node,category=QUERY,scope=httpShardHandler,name=leasedConnections, solr:dom1=core,dom2=collection1,category=QUERY,scope=/select,name=requests, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=softAutoCommits, solr:dom1=jvm,name0=memory,name1=pools,name2=Par-Survivor-Space,name=init, solr:dom1=jvm,name0=memory,name1=non-heap,name=used, solr:dom1=node,category=ADMIN,scope=/admin/metrics,name=handlerStart, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=serverErrors, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=clientErrors, java.lang:type=MemoryPool,name=Code Cache, solr:dom1=core,dom2=collection1,category=CORE,name=refCount, java.nio:type=BufferPool,name=direct, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json,name=totalTime, solr:dom1=node,category=QUERY,scope=/admin/autoscaling,name=handlerStart, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=errors, solr:dom1=jvm,name0=threads,name1=daemon,name=count, solr:dom1=node,category=ADMIN,scope=/admin/zookeeper,name=serverErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=errors, solr:dom1=node,category=ADMIN,scope=/admin/configs,name=requests, solr:dom1=jvm,name0=memory,name1=non-heap,name=init, solr:dom1=core,dom2=collection1,category=QUERY,scope=/sql,name=requestTimes, solr:dom1=core,dom2=collection1,category=CACHE,scope=searcher,name=queryResultCache, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=totalTime, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=searcherName, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH_Direct,name=handlerStart, java.lang:type=GarbageCollector,name=ConcurrentMarkSweep, solr:dom1=core,dom2=collection1,category=QUERY,scope=/dismax,name=requests, solr:dom1=jvm,name0=memory,name1=pools,name2=Par-Survivor-Space,name=used, solr:dom1=jvm,name0=memory,name1=pools,name2=Par-Survivor-Space,name=used-after-gc, solr:dom1=jvm,name0=memory,name1=heap,name=max, solr:dom1=node,category=CONTAINER,scope=threadPool,name=coreLoadExecutor.duration, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=serverErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/luke,name=errors, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=requests, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/mbeans,name=requestTimes, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/file,name=timeouts, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=indexSize, solr:dom1=node,category=ADMIN,scope=/admin/zookeeper,name=errors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=timeouts, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/graph,name=requests, solr:dom1=jvm,name0=buffers,name1=mapped,name=Count, solr:dom1=core,dom2=collection1,category=QUERY,scope=/defaults,name=errors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=requestTimes, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=timeouts, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=totalTime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH1,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/properties,name=serverErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json/docs,name=clientErrors, java.lang:type=Compilation, solr:dom1=jvm,name0=memory,name1=pools,name2=Par-Eden-Space,name=committed, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.updateExecutor.submitted, solr:dom1=node,category=QUERY,scope=/admin/metrics/collector,name=serverErrors, solr:dom1=jvm,name0=memory,name1=pools,name2=Par-Eden-Space,name=used-after-gc, solr:dom1=core,dom2=collection1,category=TLOG,scope=buffered,name=ops, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=timeouts, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=adds, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/properties,name=errors, solr:dom1=jvm,name0=os,name=version, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=serverErrors, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=pendingConnections, solr:dom1=jvm,name0=memory,name1=pools,name2=Metaspace,name=usage, solr:dom1=core,dom2=collection1,category=CACHE,scope=searcher,name=documentCache, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=totalTime, so
   [junit4]   2> lr:dom1=core,dom2=collection1,category=QUERY,scope=/graph,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mlt,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak_Direct,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/terms,name=requestTimes, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH_Direct,name=totalTime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/properties,name=totalTime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=requests, solr:dom1=jvm,name0=os,name=committedVirtualMemorySize, solr:dom1=core,dom2=collection1,category=QUERY,scope=/tvrh,name=errors, solr:dom1=node,category=QUERY,scope=/admin/autoscaling,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/terms,name=requests, solr:dom1=jvm,name0=memory,name1=pools,name2=CMS-Old-Gen,name=usage, solr:dom1=jvm,name0=memory,name1=non-heap,name=max, solr:dom1=jvm,name0=os,name=maxFileDescriptorCount, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragmentsBuilder,name=simple.requests, solr:dom1=node,category=CONTAINER,scope=threadPool,name=coreLoadExecutor.running, solr:dom1=core,dom2=collection1,category=CORE,name=coreName, solr:dom1=core,dom2=collection1,category=QUERY,scope=/sql,name=handlerStart, solr:dom1=core,dom2=collection1,category=CACHE,scope=searcher,name=perSegFilter, solr:dom1=jvm,name0=memory,name1=heap,name=used, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/logging,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/sql,name=errors, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=confFilesToReplicate, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json/docs,name=requests, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/luke,name=totalTime, solr:dom1=jvm,name0=os,name=systemCpuLoad, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragListBuilder,name=default.requests, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=splits, solr:dom1=core,dom2=collection1,category=QUERY,scope=/terms,name=handlerStart, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=fetcher, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH_Direct,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/graph,name=timeouts, solr:dom1=jvm,name0=threads,name1=blocked,name=count, solr:dom1=node,category=ADMIN,scope=/admin/zookeeper,name=requestTimes, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=totalTime, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrBoundaryScanner,name=default.requests, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=requests, solr:dom1=node,category=ADMIN,scope=/admin/configs,name=totalTime, solr:dom1=core,dom2=collection1,category=TLOG,scope=applyingBuffered,name=ops, solr:dom1=node,category=ADMIN,scope=/admin/info,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=totalTime, solr:dom1=jvm,name0=buffers,name1=direct,name=TotalCapacity, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/properties,name=clientErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=handlerStart, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=generation, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mltrh,name=errors, java.lang:type=OperatingSystem, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/luke,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/tvrh,name=handlerStart, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=totalTime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json/docs,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mock,name=clientErrors, solr:dom1=jvm,name0=threads,name1=terminated,name=count, java.lang:type=MemoryManager,name=Metaspace Manager, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/mbeans,name=handlerStart, solr:dom1=jvm,name0=memory,name1=pools,name2=Par-Eden-Space,name=used, solr:dom1=node,category=QUERY,scope=/admin/metrics/collector,name=timeouts, solr:dom1=node,category=ADMIN,scope=/admin/metrics,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH1,name=errors, solr:dom1=node,category=ADMIN,scope=/admin/zookeeper,name=handlerStart, solr:dom1=node,category=QUERY,scope=/admin/autoscaling,name=requestTimes, solr:dom1=core,dom2=collection1,category=TLOG,scope=replay,name=remaining.logs, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/system,name=requestTimes, solr:dom1=node,category=CONTAINER,scope=threadPool,name=coreContainerWorkExecutor.completed, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/properties,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/sql,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/sql,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/logging,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/luke,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH,name=totalTime, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=threadPool.parallelCoreAdminExecutor.duration, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json/docs,name=totalTime, solr:dom1=node,category=QUERY,scope=/admin/autoscaling,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak,name=serverErrors, solr:dom1=core,dom2=collection1,category=INDEX,name=sizeInBytes, solr:dom1=jvm,name0=memory,name1=pools,name2=Metaspace,name=max, solr:dom1=node,category=ADMIN,scope=/admin/info,name=requestTimes, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=isSlave, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=commits, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/sql,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=serverErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=cumulativeAdds, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mock,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/select,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mlt,name=serverErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/mbeans,name=timeouts, solr:dom1=jvm,name0=os,name=freeSwapSpaceSize, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/logging,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mltrh,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/get,name=totalTime, java.lang:type=MemoryManager,name=CodeCacheManager, solr:dom1=core,dom2=collection1,category=QUERY,scope=/stream,name=requests, solr:dom1=node,category=QUERY,scope=httpShardHandler,name=availableConnections, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=indexPath, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/debug/dump,name=requestTimes, solr:dom1=node,category=ADMIN,scope=/admin/configs,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=requests, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/debug/dump,name=totalTime, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=timeouts, solr:dom1=jvm,name0=memory,name1=pools,name2=CMS-Old-Gen,name=used-after-gc, solr:dom1=node,category=QUERY,scope=/admin/autoscaling,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/system,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/graph,name=totalTime, solr:dom1=node,category=ADMIN,scope=/admin/info,name=clientErrors, solr:dom1=jvm,name0=memory,name1=heap,name=usage, solr:dom1=core,dom2=collection1,category=INDEX,name=size, solr:dom1=jvm,name0=memory,name1=pools,name2=CMS-Old-Gen,name=init, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=requestTimes, java.nio:type=BufferPool,name=mapped, solr:dom1=jvm,name0=gc,name1=ParNew,name=count, com.sun.management:type=DiagnosticCommand, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=replicationEnabled, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/config,name=serverErrors, solr:dom1=jvm,name0=memory,name1=pools,name2=Par-Eden-Space,name=max, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=new,name=maxReached, solr:dom1=node,category=ADMIN,scope=/admin/info,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=clientErrors, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=threadPool.parallelCoreAdminExecutor.running, solr:dom1=jvm,name0=os,name=arch, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.updateExecutor.duration, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=serverErrors, solr:dom1=jvm,name0=memory,name1=pools,name2=CMS-Old-Gen,name=committed, solr:dom1=jvm,name0=os,name=name, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=serverErrors, solr:dom1=jvm,name0=os,name=processCpuLoad, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=maxDoc, solr:dom1=jvm,name0=memory,name1=pools,name2=Metaspace,name=committed, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json,name=errors, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragmentsBuilder,name=scoreOrder.requests, solr:dom1=node,category=CONTAINER,scope=fs,name=usableSpace, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH1,name=serverErrors, solr:dom1=node,category=ADMIN,scope=/admin/metrics,name=errors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=merges, solr:dom1=node,category=QUERY,scope=/admin/metrics/collector,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/tvrh,name=serverErrors, solr:dom1=node,category=CONTAINER,scope=version,name=specification, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=errors, solr:dom1=jvm,name0=threads,name1=timed_waiting,name=count, solr:dom1=core,dom2=collection1,category=CORE,name=instanceDir, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/tvrh,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/file,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/dismax,name=serverErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=handlerStart, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=deletedDocs, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json,name=clientErrors, solr:dom1=jvm,name0=memory,name1=pools,name2=Metaspace,name=used, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH1,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/config,name=requestTimes, solr:dom1=node,category=ADMIN,scope=/admin/info,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/system,name=handlerStart, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mltrh,name=serverErrors, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=readerDir, JMImplementation:type=MBeanServerDelegate, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=serverErrors, solr:dom1=node,category=ADMIN,scope=/admin/zookeeper,name=requests, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.recoveryExecutor.completed, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH,name=handlerStart, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=deletesById, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/mbeans,name=requests, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=new,name=time, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/mbeans,name=errors, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=totalTime, java.lang:type=ClassLoading, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=requests, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/file,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/select,name=requestTimes, solr:dom1=node,category=CONTAINER,scope=threadPool,name=coreContainerWorkExecutor.submitted, java.lang:type=MemoryPool,name=Metaspace, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=clientErrors, java.lang:type=GarbageCollector,name=ParNew, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=expungeDeletes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/debug/dump,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/tvrh,name=requestTimes, solr:dom1=core,dom2=collection1,category=CACHE,scope=core,name=fieldCache, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH_Direct,name=timeouts, solr:dom1=core,dom2=collection1,category=CACHE,scope=searcher,name=fieldValueCache, solr:dom1=jvm,name0=memory,name1=pools,name2=CMS-Old-Gen,name=used, solr:dom1=core,dom2=collection1,category=QUERY,scope=/terms,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/select,name=handlerStart, solr:dom1=jvm,name0=memory,name1=pools,name2=Code-Cache,name=max, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/file,name=serverErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/logging,name=clientErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=autoCommits, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=requests, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=requestTimes, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=handlerStart, solr:dom1=node,category=CONTAINER,scope=fs,name=totalSpace, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH_Direct,name=errors, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragmenter,name=default.requests, solr:dom1=jvm,name0=os,name=freePhysicalMemorySize, solr:dom1=node,category=ADMIN,scope=/admin/info,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/tvrh,name=requests, solr:dom1=node,category=ADMIN,scope=/admin/configs,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=clientErrors, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=threadPool.parallelCoreAdminExecutor.completed, solr:dom1=core,dom2=collection1,category=QUERY,scope=/defaults,name=clientErrors, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=errors, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=new,name=warmup, solr:dom1=core,dom2=collection1,category=QUERY,scope=/get,name=requests, solr:dom1=
   [junit4]   2> core,dom2=collection1,category=QUERY,scope=/mlt,name=clientErrors, java.lang:type=MemoryPool,name=Par Eden Space, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mltrh,name=timeouts, solr:dom1=node,category=QUERY,scope=httpShardHandler,name=maxConnections, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH_Direct,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/defaults,name=totalTime, solr:dom1=jvm,name0=memory,name1=pools,name2=Par-Survivor-Space,name=max, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=totalTime, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragmentsBuilder,name=default.requests, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=docsPending, solr:dom1=node,category=ADMIN,scope=/admin/zookeeper,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mock,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mlt,name=totalTime, solr:dom1=jvm,name0=system,name=properties, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=isMaster, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/logging,name=handlerStart, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=serverErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mltrh,name=requests, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrEncoder,name=default.requests, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/mbeans,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/graph,name=requestTimes, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH1,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/logging,name=requestTimes, solr:dom1=core,dom2=collection1,category=CORE,name=aliases, solr:dom1=jvm,name0=memory,name1=total,name=max, solr:dom1=jvm,name0=memory,name1=pools,name2=Code-Cache,name=used, solr:dom1=core,dom2=collection1,category=QUERY,scope=/select,name=errors, solr:dom1=jvm,name0=memory,name1=pools,name2=Par-Eden-Space,name=init, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/debug/dump,name=errors, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=requests, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=registeredAt, solr:dom1=core,dom2=collection1,category=QUERY,scope=/get,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/get,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/file,name=requests, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=replicateAfter, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mltrh,name=handlerStart, solr:dom1=core,dom2=collection1,category=CACHE,scope=searcher,name=filterCache, solr:dom1=node,category=ADMIN,scope=/admin/info,name=serverErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=handlerStart, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/config,name=timeouts, solr:dom1=core,dom2=collection1,category=CORE,name=startTime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json/docs,name=requestTimes, solr:dom1=node,category=CONTAINER,scope=threadPool,name=coreLoadExecutor.completed, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/properties,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=requestTimes, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=requestTimes, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrBoundaryScanner,name=simple.requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/defaults,name=requests, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/file,name=requestTimes, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/select,name=serverErrors, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=maxConnections, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mltrh,name=requestTimes, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=availableConnections, solr:dom1=core,dom2=collection1,category=SEARCHER,name=new, solr:dom1=node,category=ADMIN,scope=/admin/configs,name=serverErrors, solr:dom1=node,category=CONTAINER,scope=version,name=implementation, com.sun.management:type=HotSpotDiagnostic, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mlt,name=requests, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/luke,name=clientErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json/docs,name=errors, solr:dom1=node,category=CONTAINER,scope=threadPool,name=coreLoadExecutor.submitted, solr:dom1=jvm,name0=memory,name1=non-heap,name=committed, solr:dom1=jvm,name0=buffers,name1=mapped,name=MemoryUsed, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/terms,name=errors, java.util.logging:type=Logging, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/system,name=clientErrors, solr:dom1=jvm,name0=memory,name1=pools,name2=Metaspace,name=init, solr:dom1=core,dom2=collection1,category=QUERY,scope=/dismax,name=totalTime, solr:dom1=jvm,name0=buffers,name1=direct,name=Count, solr:dom1=core,dom2=collection1,category=QUERY,scope=/select,name=totalTime, solr:dom1=jvm,name0=os,name=processCpuTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/get,name=serverErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/dismax,name=requestTimes, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/system,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/config,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/stream,name=errors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=requestTimes, solr:dom1=jvm,name0=memory,name1=heap,name=committed, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/file,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/defaults,name=requestTimes, solr:dom1=jvm,name0=threads,name1=deadlock,name=count, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH1,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/graph,name=errors, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=warmupTime, solr:dom1=jvm,name0=threads,name1=runnable,name=count, solr:dom1=node,category=ADMIN,scope=/admin/metrics,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/config,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak_Direct,name=requestTimes, solr:dom1=jvm,name0=gc,name1=ConcurrentMarkSweep,name=time, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=indexVersion, solr:dom1=jvm,name0=memory,name1=heap,name=init, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=numDocs, java.lang:type=MemoryPool,name=CMS Old Gen, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak,name=requestTimes, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=serverErrors, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFormatter,name=default.requests, solr:dom1=core,dom2=collection1,category=CORE,name=indexDir, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=errors, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/dismax,name=errors, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragmenter,name=regex.requests, solr:dom1=node,category=ADMIN,scope=/admin/configs,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/get,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/select,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/config,name=requests, solr:dom1=node,category=QUERY,scope=/admin/metrics/collector,name=clientErrors, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.updateExecutor.completed, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/get,name=clientErrors, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=errors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json,name=handlerStart, solr:dom1=node,category=QUERY,scope=httpShardHandler,name=threadPool.httpShardExecutor.submitted, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=timeouts, solr:dom1=node,category=QUERY,scope=httpShardHandler,name=threadPool.httpShardExecutor.running, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/system,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/debug/dump,name=handlerStart, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/system,name=requests, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=totalTime, solr:dom1=jvm,name0=memory,name1=pools,name2=Par-Survivor-Space,name=usage, solr:dom1=core,dom2=collection1,category=QUERY,scope=/debug/dump,name=requests, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json/docs,name=handlerStart, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/luke,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=handlerStart, solr:dom1=jvm,name0=memory,name1=pools,name2=CMS-Old-Gen,name=max, solr:dom1=node,category=CONTAINER,scope=cores,name=lazy, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=requests, solr:dom1=node,category=QUERY,scope=/admin/metrics/collector,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/defaults,name=handlerStart, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/properties,name=requestTimes, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=optimizes, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=indexVersion, solr:dom1=core,dom2=collection1,category=QUERY,scope=/tvrh,name=totalTime, solr:dom1=jvm,name0=os,name=openFileDescriptorCount, solr:dom1=node,category=QUERY,scope=httpShardHandler,name=pendingConnections, solr:dom1=jvm,name0=memory,name1=pools,name2=Code-Cache,name=committed, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mlt,name=timeouts, solr:dom1=node,category=ADMIN,scope=/admin/configs,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=errors, solr:dom1=node,category=CACHE,name=fieldCache, solr:dom1=core,dom2=collection1,category=QUERY,scope=/stream,name=handlerStart, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/mbeans,name=totalTime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak_Direct,name=totalTime, solr:dom1=jvm,name0=memory,name1=total,name=committed, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mock,name=requestTimes, solr:dom1=node,category=QUERY,scope=/admin/metrics/collector,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/debug/dump,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/stream,name=serverErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/system,name=serverErrors, solr:dom1=jvm,name0=threads,name=deadlocks, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/config,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak_Direct,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH_Direct,name=serverErrors, solr:dom1=jvm,name0=memory,name1=non-heap,name=usage, solr:dom1=jvm,name0=classes,name=loaded, solr:dom1=core,dom2=collection1,category=QUERY,scope=/tvrh,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/stream,name=totalTime, java.lang:type=Memory, solr:dom1=jvm,name0=memory,name1=total,name=init, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/logging,name=errors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/luke,name=requestTimes, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=leasedConnections, solr:dom1=node,category=ADMIN,scope=/admin/info,name=errors, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragListBuilder,name=simple.requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH,name=timeouts]
   [junit4]   2> 1556118 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[F830FF2FC67C0210]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1 (1573157578966302720)]} 0 0
   [junit4]   2> 1556120 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[F830FF2FC67C0210]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1573157578969448448,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1556120 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[F830FF2FC67C0210]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@43928e8c commitCommandVersion:1573157578969448448
   [junit4]   2> 1556129 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[F830FF2FC67C0210]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1b989ff9[collection1] main]
   [junit4]   2> 1556129 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[F830FF2FC67C0210]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1556131 INFO  (searcherExecutor-8158-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1b989ff9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):c1)))}
   [junit4]   2> 1556131 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[F830FF2FC67C0210]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 11
   [junit4]   2> 1556131 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[F830FF2FC67C0210]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testJmxUpdate
   [junit4]   2> 1556131 INFO  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 1556131 INFO  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1857328979
   [junit4]   2> 1556131 INFO  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1556132 INFO  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1556133 INFO  (SUITE-TestJmxIntegration-seed#[F830FF2FC67C0210]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1556133 INFO  (coreCloseExecutor-8163-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@41b4f5b9
   [junit4]   2> 1556135 INFO  (coreCloseExecutor-8163-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1102378425
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.core.TestJmxIntegration_F830FF2FC67C0210-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {range_facet_l_dv=FSTOrd50, multiDefault=BlockTreeOrds(blocksize=128), intDefault=PostingsFormat(name=LuceneVarGapFixedInterval), id_i1=PostingsFormat(name=LuceneVarGapFixedInterval), id=FSTOrd50, range_facet_i_dv=BlockTreeOrds(blocksize=128), range_facet_l=BlockTreeOrds(blocksize=128), timestamp=BlockTreeOrds(blocksize=128)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene70), _version_=DocValuesFormat(name=Lucene70), intDefault=DocValuesFormat(name=Lucene70), id_i1=DocValuesFormat(name=Lucene70), range_facet_i_dv=DocValuesFormat(name=Direct), intDvoDefault=DocValuesFormat(name=Lucene70), range_facet_l=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=1264, maxMBSortInHeap=7.850646354664804, sim=RandomSimilarity(queryNorm=false): {}, locale=vi-VN, timezone=America/Kralendijk
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=3,threads=1,free=306458768,total=417325056
   [junit4]   2> NOTE: All tests run in this JVM: [ZkStateReaderTest, FileUtilsTest, TestChildDocTransformer, TestSearchPerf, ConnectionManagerTest, TestSolrQueryParser, RulesTest, TestOnReconnectListenerSupport, TestSuggestSpellingConverter, TestSchemaSimilarityResource, TestRealTimeGet, QueryEqualityTest, TermVectorComponentTest, TestSolrCoreSnapshots, CloudExitableDirectoryReaderTest, SolrIndexConfigTest, ResponseHeaderTest, BlockCacheTest, JvmMetricsTest, TestCustomStream, TestIntervalFaceting, TestSchemaManager, TestRebalanceLeaders, TestTrie, TestXmlQParser, TestNamedUpdateProcessors, StatsComponentTest, TestSchemaNameResource, TestBadConfig, BigEndianAscendingWordSerializerTest, SchemaVersionSpecificBehaviorTest, AutoScalingHandlerTest, DistributedFacetPivotWhiteBoxTest, VersionInfoTest, TestLegacyFieldCache, DocumentBuilderTest, SpellingQueryConverterTest, TestCloudInspectUtil, AtomicUpdateProcessorFactoryTest, CollectionsAPIDistributedZkTest, TestFieldCache, TestPartialUpdateDeduplication, UUIDUpdateProcessorFallbackTest, HdfsChaosMonkeyNothingIsSafeTest, DeleteStatusTest, AddBlockUpdateTest, ManagedSchemaRoundRobinCloudTest, TestCoreDiscovery, DistributedTermsComponentTest, TestExactSharedStatsCache, NodeMutatorTest, StressHdfsTest, TestSystemIdResolver, TestRandomFaceting, TestAuthenticationFramework, DeleteLastCustomShardedReplicaTest, UninvertDocValuesMergePolicyTest, PreAnalyzedFieldTest, BlockJoinFacetDistribTest, DistribDocExpirationUpdateProcessorTest, TestSolrCloudWithHadoopAuthPlugin, TestSubQueryTransformerCrossCore, TestManagedSynonymGraphFilterFactory, TestRestoreCore, TermsComponentTest, TestFreeTextSuggestions, TestCloudRecovery, LoggingHandlerTest, TestReplicationHandlerBackup, SignatureUpdateProcessorFactoryTest, SparseHLLTest, PluginInfoTest, ZkSolrClientTest, HdfsCollectionsAPIDistributedZkTest, TestConfig, TestExportWriter, ExplicitHLLTest, TestPerFieldSimilarity, EchoParamsTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, SuggestComponentTest, TestFastWriter, SolrCmdDistributorTest, TestStressLucene, OverseerModifyCollectionTest, CdcrReplicationDistributedZkTest, TestClassNameShortening, BasicZkTest, TriLevelCompositeIdRoutingTest, TestRemoteStreaming, TestDFISimilarityFactory, TestReload, SpellCheckCollatorTest, SolrCoreMetricManagerTest, IndexSchemaRuntimeFieldTest, SimpleMLTQParserTest, TestCloudNestedDocsSort, TestLeaderInitiatedRecoveryThread, TestQueryTypes, HdfsBasicDistributedZk2Test, TestTlogReplica, FieldMutatingUpdateProcessorTest, TestMaxTokenLenTokenizer, ReplicaListTransformerTest, TestMinMaxOnMultiValuedField, CopyFieldTest, TestManagedSchemaThreadSafety, TestSolrJ, TestConfigSets, TestBM25SimilarityFactory, TestLRUCache, TestCollectionAPI, BaseCdcrDistributedZkTest, TestQueryUtils, ConcurrentDeleteAndCreateCollectionTest, TestCollapseQParserPlugin, TestRestManager, HttpPartitionTest, PeerSyncWithIndexFingerprintCachingTest, DirectoryFactoryTest, CoreMergeIndexesAdminHandlerTest, TestStressUserVersions, SpellPossibilityIteratorTest, TestFieldCacheVsDocValues, TestRawTransformer, RollingRestartTest, TestLeaderElectionWithEmptyReplica, TestJettySolrRunner, SolrCloudExampleTest, TestManagedSchema, DebugComponentTest, TestAuthorizationFramework, TestMultiWordSynonyms, TestNumericRangeQuery32, TestElisionMultitermQuery, DistributedExpandComponentTest, TestDocumentBuilder, TestSimpleTrackingShardHandler, ImplicitSnitchTest, TestDistributedGrouping, SolrCloudReportersTest, PeerSyncReplicationTest, TestDelegationWithHadoopAuth, ExternalFileFieldSortTest, TestManagedResource, TermVectorComponentDistributedTest, AssignTest, TestGraphTermsQParserPlugin, MergeStrategyTest, JavabinLoaderTest, FacetPivotSmallTest, MoveReplicaHDFSTest, DistributedIntervalFacetingTest, DistributedQueueTest, TestConfigReload, TestSSLRandomization, TestDynamicFieldCollectionResource, StatsReloadRaceTest, HdfsRestartWhileUpdatingTest, MoreLikeThisHandlerTest, LeaderElectionIntegrationTest, TestSolrConfigHandlerConcurrent, ConfigureRecoveryStrategyTest, SpellCheckComponentTest, HdfsThreadLeakTest, TestDefaultStatsCache, TestMaxScoreQueryParser, TestCharFilters, TestIndexSearcher, BufferStoreTest, TestCSVLoader, BasicAuthStandaloneTest, OverriddenZkACLAndCredentialsProvidersTest, DocExpirationUpdateProcessorFactoryTest, TestRequestForwarding, RemoteQueryErrorTest, PolyFieldTest, OverseerTest, TestReplicationHandler, BadIndexSchemaTest, ConvertedLegacyTest, TestFiltering, BasicFunctionalityTest, TestLazyCores, ShowFileRequestHandlerTest, DistributedQueryElevationComponentTest, SimplePostToolTest, AnalysisAfterCoreReloadTest, SuggesterFSTTest, SolrRequestParserTest, TestFoldingMultitermQuery, SuggesterTSTTest, SuggesterTest, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, SolrCoreCheckLockOnStartupTest, TestUpdate, TestWordDelimiterFilterFactory, StatelessScriptUpdateProcessorFactoryTest, DocValuesMultiTest, TestSolrDeletionPolicy1, SolrInfoBeanTest, XsltUpdateRequestHandlerTest, CacheHeaderTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, LukeRequestHandlerTest, TestWriterPerf, DirectSolrSpellCheckerTest, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, PrimitiveFieldTypeTest, TestOmitPositions, FileBasedSpellCheckerTest, XmlUpdateRequestHandlerTest, TestValueSourceCache, PathHierarchyTokenizerFactoryTest, TestIndexingPerformance, RequiredFieldsTest, SolrPluginUtilsTest, IndexSchemaTest, TestJmxIntegration]
   [junit4] Completed [432/728 (1!)] on J1 in 0.39s, 3 tests, 1 error, 1 skipped <<< FAILURES!

[...truncated 44333 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 48 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/48/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.MoveReplicaHDFSUlogDirTest.testDataDirAndUlogAreMaintained

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:57615/solr

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:57615/solr
        at __randomizedtesting.SeedInfo.seed([143E50FB78395A27:3B0DC528C449E805]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:637)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:252)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
        at org.apache.solr.cloud.MoveReplicaHDFSUlogDirTest.testDataDirAndUlogAreMaintained(MoveReplicaHDFSUlogDirTest.java:84)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:171)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139)
        at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155)
        at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
        at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
        at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165)
        at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167)
        at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:523)
        ... 49 more




Build Log:
[...truncated 11113 lines...]
   [junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSUlogDirTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSUlogDirTest_143E50FB78395A27-001/init-core-data-001
   [junit4]   2> 536139 WARN  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
   [junit4]   2> 536139 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 536140 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 536140 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSUlogDirTest_143E50FB78395A27-001/tempDir-001
   [junit4]   2> 536140 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 536140 INFO  (Thread-1170) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 536140 INFO  (Thread-1170) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 536142 ERROR (Thread-1170) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 536240 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:33625
   [junit4]   2> 536247 INFO  (jetty-launcher-1043-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 536247 INFO  (jetty-launcher-1043-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 536249 INFO  (jetty-launcher-1043-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7de45730{/solr,null,AVAILABLE}
   [junit4]   2> 536250 INFO  (jetty-launcher-1043-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@16341d85{/solr,null,AVAILABLE}
   [junit4]   2> 536253 INFO  (jetty-launcher-1043-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7fffac00{HTTP/1.1,[http/1.1]}{127.0.0.1:63691}
   [junit4]   2> 536253 INFO  (jetty-launcher-1043-thread-1) [    ] o.e.j.s.Server Started @538810ms
   [junit4]   2> 536253 INFO  (jetty-launcher-1043-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=63691}
   [junit4]   2> 536253 INFO  (jetty-launcher-1043-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@585f4f6{HTTP/1.1,[http/1.1]}{127.0.0.1:57615}
   [junit4]   2> 536253 INFO  (jetty-launcher-1043-thread-2) [    ] o.e.j.s.Server Started @538810ms
   [junit4]   2> 536253 INFO  (jetty-launcher-1043-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57615}
   [junit4]   2> 536254 ERROR (jetty-launcher-1043-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 536254 ERROR (jetty-launcher-1043-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 536254 INFO  (jetty-launcher-1043-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.1.0
   [junit4]   2> 536254 INFO  (jetty-launcher-1043-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.1.0
   [junit4]   2> 536254 INFO  (jetty-launcher-1043-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 536254 INFO  (jetty-launcher-1043-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 536254 INFO  (jetty-launcher-1043-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 536254 INFO  (jetty-launcher-1043-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 536254 INFO  (jetty-launcher-1043-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-17T20:27:30.700Z
   [junit4]   2> 536254 INFO  (jetty-launcher-1043-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-17T20:27:30.700Z
   [junit4]   2> 536257 INFO  (jetty-launcher-1043-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 536257 INFO  (jetty-launcher-1043-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 536268 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15d523c019b0001, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 536268 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15d523c019b0002, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 536270 INFO  (jetty-launcher-1043-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33625/solr
   [junit4]   2> 536270 INFO  (jetty-launcher-1043-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33625/solr
   [junit4]   2> 536412 INFO  (jetty-launcher-1043-thread-2) [n:127.0.0.1:57615_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 536412 INFO  (jetty-launcher-1043-thread-2) [n:127.0.0.1:57615_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57615_solr
   [junit4]   2> 536414 INFO  (jetty-launcher-1043-thread-2) [n:127.0.0.1:57615_solr    ] o.a.s.c.Overseer Overseer (id=98325184550469638-127.0.0.1:57615_solr-n_0000000000) starting
   [junit4]   2> 536423 INFO  (jetty-launcher-1043-thread-2) [n:127.0.0.1:57615_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57615_solr
   [junit4]   2> 536426 INFO  (zkCallback-1055-thread-1-processing-n:127.0.0.1:57615_solr) [n:127.0.0.1:57615_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 536427 INFO  (jetty-launcher-1043-thread-1) [n:127.0.0.1:63691_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 536428 INFO  (zkCallback-1054-thread-1-processing-n:127.0.0.1:63691_solr) [n:127.0.0.1:63691_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 536430 INFO  (jetty-launcher-1043-thread-1) [n:127.0.0.1:63691_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63691_solr
   [junit4]   2> 536432 INFO  (zkCallback-1055-thread-1-processing-n:127.0.0.1:57615_solr) [n:127.0.0.1:57615_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 536432 INFO  (zkCallback-1054-thread-1-processing-n:127.0.0.1:63691_solr) [n:127.0.0.1:63691_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 536501 INFO  (jetty-launcher-1043-thread-2) [n:127.0.0.1:57615_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57615.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ce0ef48
   [junit4]   2> 536503 INFO  (jetty-launcher-1043-thread-2) [n:127.0.0.1:57615_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57615.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ce0ef48
   [junit4]   2> 536503 INFO  (jetty-launcher-1043-thread-2) [n:127.0.0.1:57615_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57615.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ce0ef48
   [junit4]   2> 536506 INFO  (jetty-launcher-1043-thread-2) [n:127.0.0.1:57615_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSUlogDirTest_143E50FB78395A27-001/tempDir-001/node2/.
   [junit4]   2> 536508 INFO  (jetty-launcher-1043-thread-1) [n:127.0.0.1:63691_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63691.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ce0ef48
   [junit4]   2> 536510 INFO  (jetty-launcher-1043-thread-1) [n:127.0.0.1:63691_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63691.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ce0ef48
   [junit4]   2> 536511 INFO  (jetty-launcher-1043-thread-1) [n:127.0.0.1:63691_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63691.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ce0ef48
   [junit4]   2> 536511 INFO  (jetty-launcher-1043-thread-1) [n:127.0.0.1:63691_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSUlogDirTest_143E50FB78395A27-001/tempDir-001/node1/.
   [junit4]   2> 536531 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 536532 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33625/solr ready
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 536556 WARN  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 536560 WARN  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 536561 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 536571 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.m.log Extract jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_serv1.vm3_35888_hdfs____.mpyko9/webapp
   [junit4]   2> 537045 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@serv1-vm3:35888
   [junit4]   2> 537132 WARN  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 537133 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 537142 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.m.log Extract jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_58087_datanode____96jij7/webapp
   [junit4]   2> 537591 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58087
   [junit4]   2> 537648 WARN  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 537649 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 537660 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.m.log Extract jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_48089_datanode____.mwjwbw/webapp
   [junit4]   2> 537698 INFO  (IPC Server handler 4 on 41607) [    ] BlockStateChange BLOCK* processReport: from storage DS-b843a369-6749-43c7-a36f-f8b2bf98bcc2 node DatanodeRegistration(127.0.0.1:34260, datanodeUuid=c6b177a8-0655-461b-8cf7-1ae8ba85fabd, infoPort=46593, infoSecurePort=0, ipcPort=60838, storageInfo=lv=-56;cid=testClusterID;nsid=1822056119;c=0), blocks: 0, hasStaleStorage: true, processing time: 1 msecs
   [junit4]   2> 537699 INFO  (IPC Server handler 4 on 41607) [    ] BlockStateChange BLOCK* processReport: from storage DS-24967e7a-4fc4-4de9-89ac-2bb87eb298a8 node DatanodeRegistration(127.0.0.1:34260, datanodeUuid=c6b177a8-0655-461b-8cf7-1ae8ba85fabd, infoPort=46593, infoSecurePort=0, ipcPort=60838, storageInfo=lv=-56;cid=testClusterID;nsid=1822056119;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 538143 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:48089
   [junit4]   2> 538254 INFO  (IPC Server handler 6 on 41607) [    ] BlockStateChange BLOCK* processReport: from storage DS-0a093921-a778-408d-9c84-5d41bbcbac9c node DatanodeRegistration(127.0.0.1:39803, datanodeUuid=dbe974f9-895f-4d18-9b56-35e6b31fe2b7, infoPort=41861, infoSecurePort=0, ipcPort=40458, storageInfo=lv=-56;cid=testClusterID;nsid=1822056119;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 538254 INFO  (IPC Server handler 6 on 41607) [    ] BlockStateChange BLOCK* processReport: from storage DS-c5ea3bfe-2eab-408c-94b9-48a3abadc4d9 node DatanodeRegistration(127.0.0.1:39803, datanodeUuid=dbe974f9-895f-4d18-9b56-35e6b31fe2b7, infoPort=41861, infoSecurePort=0, ipcPort=40458, storageInfo=lv=-56;cid=testClusterID;nsid=1822056119;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 538339 INFO  (TEST-MoveReplicaHDFSUlogDirTest.testDataDirAndUlogAreMaintained-seed#[143E50FB78395A27]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDataDirAndUlogAreMaintained
   [junit4]   2> 538341 INFO  (qtp359362206-5847) [n:127.0.0.1:57615_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=movereplicatest_coll2&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 538344 INFO  (OverseerThreadFactory-1909-thread-1-processing-n:127.0.0.1:57615_solr) [n:127.0.0.1:57615_solr    ] o.a.s.c.CreateCollectionCmd Create collection movereplicatest_coll2
   [junit4]   2> 538344 WARN  (OverseerThreadFactory-1909-thread-1-processing-n:127.0.0.1:57615_solr) [n:127.0.0.1:57615_solr    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (movereplicatest_coll2) without cores.
   [junit4]   2> 538458 INFO  (qtp359362206-5847) [n:127.0.0.1:57615_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 538459 INFO  (qtp359362206-5847) [n:127.0.0.1:57615_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=movereplicatest_coll2&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&wt=javabin&version=2} status=0 QTime=117
   [junit4]   2> 538460 INFO  (qtp359362206-5848) [n:127.0.0.1:57615_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:57615_solr&dataDir=hdfs://serv1-vm3:41607/dummyFolder/dataDir&action=ADDREPLICA&collection=movereplicatest_coll2&shard=shard1&type=NRT&wt=javabin&version=2&ulogDir=hdfs://serv1-vm3:41607/dummyFolder2/ulogDir and sendToOCPQueue=true
   [junit4]   2> 538468 INFO  (OverseerCollectionConfigSetProcessor-98325184550469638-127.0.0.1:57615_solr-n_0000000000) [n:127.0.0.1:57615_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 538469 INFO  (OverseerThreadFactory-1909-thread-2-processing-n:127.0.0.1:57615_solr) [n:127.0.0.1:57615_solr    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:57615_solr for creating new replica
   [junit4]   2> 538474 INFO  (OverseerStateUpdate-98325184550469638-127.0.0.1:57615_solr-n_0000000000) [n:127.0.0.1:57615_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"addreplica",
   [junit4]   2>   "collection":"movereplicatest_coll2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"movereplicatest_coll2_shard1_replica_n0",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:57615/solr",
   [junit4]   2>   "node_name":"127.0.0.1:57615_solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 538684 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node1&dataDir=hdfs://serv1-vm3:41607/dummyFolder/dataDir&collection.configName=conf1&name=movereplicatest_coll2_shard1_replica_n0&action=CREATE&collection=movereplicatest_coll2&shard=shard1&wt=javabin&version=2&ulogDir=hdfs://serv1-vm3:41607/dummyFolder2/ulogDir&replicaType=NRT
   [junit4]   2> 538689 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 538706 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 538711 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.s.IndexSchema [movereplicatest_coll2_shard1_replica_n0] Schema name=minimal
   [junit4]   2> 538712 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 538712 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.c.CoreContainer Creating SolrCore 'movereplicatest_coll2_shard1_replica_n0' using configuration from collection movereplicatest_coll2, trusted=true
   [junit4]   2> 538713 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57615.solr.core.movereplicatest_coll2.shard1.replica_n0' (registry 'solr.core.movereplicatest_coll2.shard1.replica_n0') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ce0ef48
   [junit4]   2> 538717 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://serv1-vm3:41607/data
   [junit4]   2> 538717 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 538717 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 538717 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.c.SolrCore [[movereplicatest_coll2_shard1_replica_n0] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSUlogDirTest_143E50FB78395A27-001/tempDir-001/node2/movereplicatest_coll2_shard1_replica_n0], dataDir=[hdfs://serv1-vm3:41607/dummyFolder/dataDir/]
   [junit4]   2> 538718 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://serv1-vm3:41607/dummyFolder/dataDir/snapshot_metadata
   [junit4]   2> 538725 WARN  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 538803 INFO  (zkCallback-1055-thread-1-processing-n:127.0.0.1:57615_solr) [n:127.0.0.1:57615_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/movereplicatest_coll2/state.json] for collection [movereplicatest_coll2] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 538804 INFO  (zkCallback-1055-thread-2-processing-n:127.0.0.1:57615_solr) [n:127.0.0.1:57615_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/movereplicatest_coll2/state.json] for collection [movereplicatest_coll2] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 543731 WARN  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 545546 WARN  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 550867 ERROR (OverseerThreadFactory-1909-thread-2-processing-n:127.0.0.1:57615_solr) [n:127.0.0.1:57615_solr    ] o.a.s.c.OverseerCollectionMessageHandler Error from shard: http://127.0.0.1:57615/solr
   [junit4]   2> org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:57615/solr
   [junit4]   2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:637)
   [junit4]   2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:252)
   [junit4]   2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:172)
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]   2> at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]   2> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
   [junit4]   2> at java.net.SocketInputStream.read(SocketInputStream.java:171)
   [junit4]   2> at java.net.SocketInputStream.read(SocketInputStream.java:141)
   [junit4]   2> at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139)
   [junit4]   2> at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155)
   [junit4]   2> at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284)
   [junit4]   2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]   2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]   2> at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]   2> at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165)
   [junit4]   2> at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167)
   [junit4]   2> at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]   2> at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]   2> at org.apache.solr.util.stats.InstrumentedHttpRequestExecutor.execute(InstrumentedHttpRequestExecutor.java:118)
   [junit4]   2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
   [junit4]   2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
   [junit4]   2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
   [junit4]   2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
   [junit4]   2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
   [junit4]   2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]   2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]   2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:523)
   [junit4]   2> ... 12 more
   [junit4]   2> 550868 INFO  (TEST-MoveReplicaHDFSUlogDirTest.testDataDirAndUlogAreMaintained-seed#[143E50FB78395A27]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDataDirAndUlogAreMaintained
   [junit4]   2> 550873 ERROR (OverseerThreadFactory-1909-thread-2-processing-n:127.0.0.1:57615_solr) [n:127.0.0.1:57615_solr    ] o.a.s.c.OverseerCollectionMessageHandler Collection: movereplicatest_coll2 operation: addreplica failed:org.apache.solr.common.SolrException: ADDREPLICA failed to create replica
   [junit4]   2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processResponses(OverseerCollectionMessageHandler.java:738)
   [junit4]   2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processResponses(OverseerCollectionMessageHandler.java:721)
   [junit4]   2> at org.apache.solr.cloud.AddReplicaCmd.lambda$addReplica$0(AddReplicaCmd.java:181)
   [junit4]   2> at org.apache.solr.cloud.AddReplicaCmd.addReplica(AddReplicaCmd.java:187)
   [junit4]   2> at org.apache.solr.cloud.AddReplicaCmd.call(AddReplicaCmd.java:66)
   [junit4]   2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:227)
   [junit4]   2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:464)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:57615/solr
   [junit4]   2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:637)
   [junit4]   2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:252)
   [junit4]   2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:172)
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
   [junit4]   2> ... 4 more
   [junit4]   2> Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]   2> at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]   2> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
   [junit4]   2> at java.net.SocketInputStream.read(SocketInputStream.java:171)
   [junit4]   2> at java.net.SocketInputStream.read(SocketInputStream.java:141)
   [junit4]   2> at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139)
   [junit4]   2> at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155)
   [junit4]   2> at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284)
   [junit4]   2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]   2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]   2> at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]   2> at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165)
   [junit4]   2> at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167)
   [junit4]   2> at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]   2> at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]   2> at org.apache.solr.util.stats.InstrumentedHttpRequestExecutor.execute(InstrumentedHttpRequestExecutor.java:118)
   [junit4]   2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
   [junit4]   2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
   [junit4]   2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
   [junit4]   2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
   [junit4]   2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
   [junit4]   2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]   2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]   2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:523)
   [junit4]   2> ... 12 more
   [junit4]   2>
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=MoveReplicaHDFSUlogDirTest -Dtests.method=testDataDirAndUlogAreMaintained -Dtests.seed=143E50FB78395A27 -Dtests.slow=true -Dtests.locale=fr-BE -Dtests.timezone=Hongkong -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   12.6s J0 | MoveReplicaHDFSUlogDirTest.testDataDirAndUlogAreMaintained <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:57615/solr
   [junit4]    > at __randomizedtesting.SeedInfo.seed([143E50FB78395A27:3B0DC528C449E805]:0)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:637)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:252)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
   [junit4]    > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
   [junit4]    > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
   [junit4]    > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
   [junit4]    > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
   [junit4]    > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
   [junit4]    > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
   [junit4]    > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
   [junit4]    > at org.apache.solr.cloud.MoveReplicaHDFSUlogDirTest.testDataDirAndUlogAreMaintained(MoveReplicaHDFSUlogDirTest.java:84)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
   [junit4]    > at java.net.SocketInputStream.read(SocketInputStream.java:171)
   [junit4]    > at java.net.SocketInputStream.read(SocketInputStream.java:141)
   [junit4]    > at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139)
   [junit4]    > at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155)
   [junit4]    > at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284)
   [junit4]    > at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
   [junit4]    > at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165)
   [junit4]    > at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167)
   [junit4]    > at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
   [junit4]    > at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
   [junit4]    > at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
   [junit4]    > at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
   [junit4]    > at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
   [junit4]    > at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
   [junit4]    > at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
   [junit4]    > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:523)
   [junit4]    > ... 49 more
   [junit4]   2> 550883 INFO  (jetty-closer-1044-thread-2) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@7fffac00{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 550885 INFO  (jetty-closer-1044-thread-2) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1517042796
   [junit4]   2> 550885 INFO  (jetty-closer-1044-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 550888 INFO  (qtp359362206-5848) [n:127.0.0.1:57615_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:57615_solr&dataDir=hdfs://serv1-vm3:41607/dummyFolder/dataDir&action=ADDREPLICA&collection=movereplicatest_coll2&shard=shard1&type=NRT&wt=javabin&version=2&ulogDir=hdfs://serv1-vm3:41607/dummyFolder2/ulogDir} status=500 QTime=15301
   [junit4]   2> 550888 INFO  (jetty-closer-1044-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 550888 ERROR (qtp359362206-5848) [n:127.0.0.1:57615_solr    ] o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: ADDREPLICA failed to create replica
   [junit4]   2> at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:295)
   [junit4]   2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:232)
   [junit4]   2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:210)
   [junit4]   2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:745)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:726)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:507)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
   [junit4]   2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>
   [junit4]   2> 550889 INFO  (jetty-closer-1044-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@585f4f6{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 550890 INFO  (jetty-closer-1044-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 550891 INFO  (jetty-closer-1044-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=377793164
   [junit4]   2> 550891 INFO  (jetty-closer-1044-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 550893 INFO  (jetty-closer-1044-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 550895 INFO  (jetty-closer-1044-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 550897 INFO  (jetty-closer-1044-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 550898 INFO  (jetty-closer-1044-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 550898 INFO  (jetty-closer-1044-thread-2) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 550900 INFO  (zkCallback-1055-thread-1-processing-n:127.0.0.1:57615_solr) [n:127.0.0.1:57615_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 550901 INFO  (jetty-closer-1044-thread-1) [    ] o.a.s.c.Overseer Overseer (id=98325184550469638-127.0.0.1:57615_solr-n_0000000000) closing
   [junit4]   2> 550901 INFO  (OverseerStateUpdate-98325184550469638-127.0.0.1:57615_solr-n_0000000000) [n:127.0.0.1:57615_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:57615_solr
   [junit4]   2> 550904 INFO  (jetty-closer-1044-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7de45730{/solr,null,UNAVAILABLE}
   [junit4]   2> 550912 INFO  (jetty-closer-1044-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@16341d85{/solr,null,UNAVAILABLE}
   [junit4]   2> 550925 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://serv1-vm3:41607/dummyFolder/dataDir
   [junit4]   2> 550941 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://serv1-vm3:41607/dummyFolder/dataDir/index
   [junit4]   2> 551002 INFO  (IPC Server handler 7 on 41607) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34260 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-c5ea3bfe-2eab-408c-94b9-48a3abadc4d9:NORMAL:127.0.0.1:39803|RBW], ReplicaUC[[DISK]DS-24967e7a-4fc4-4de9-89ac-2bb87eb298a8:NORMAL:127.0.0.1:34260|RBW]]} size 0
   [junit4]   2> 551002 INFO  (IPC Server handler 4 on 41607) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:39803 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-c5ea3bfe-2eab-408c-94b9-48a3abadc4d9:NORMAL:127.0.0.1:39803|RBW], ReplicaUC[[DISK]DS-24967e7a-4fc4-4de9-89ac-2bb87eb298a8:NORMAL:127.0.0.1:34260|RBW]]} size 0
   [junit4]   2> 551066 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 551066 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 551066 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
   [junit4]   2> 551078 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 551078 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 551106 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.s.SolrIndexSearcher Opening [Searcher@7453066b[movereplicatest_coll2_shard1_replica_n0] main]
   [junit4]   2> 551107 ERROR (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.r.ManagedResourceStorage Failed to get config name due to
   [junit4]   2> org.apache.solr.common.SolrException: Error loading config name for collection movereplicatest_coll2
   [junit4]   2> at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:206)
   [junit4]   2> at org.apache.solr.rest.ManagedResourceStorage.newStorageIO(ManagedResourceStorage.java:95)
   [junit4]   2> at org.apache.solr.core.SolrCore.initRestManager(SolrCore.java:2768)
   [junit4]   2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:963)
   [junit4]   2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)
   [junit4]   2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:969)
   [junit4]   2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:904)
   [junit4]   2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
   [junit4]   2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
   [junit4]   2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
   [junit4]   2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
   [junit4]   2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:745)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:726)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:507)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
   [junit4]   2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/movereplicatest_coll2
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
   [junit4]   2> at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:186)
   [junit4]   2> ... 40 more
   [junit4]   2> 551107 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.c.SolrCore [movereplicatest_coll2_shard1_replica_n0]  CLOSING SolrCore org.apache.solr.core.SolrCore@4a3c682a
   [junit4]   2> 551109 INFO  (searcherExecutor-1914-thread-1-processing-n:127.0.0.1:57615_solr x:movereplicatest_coll2_shard1_replica_n0 s:shard1 c:movereplicatest_coll2 r:core_node1) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.c.SolrCore [movereplicatest_coll2_shard1_replica_n0] Registered new searcher Searcher@7453066b[movereplicatest_coll2_shard1_replica_n0] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 551113 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.movereplicatest_coll2.shard1.replica_n0, tag=1245472810
   [junit4]   2> 551116 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.movereplicatest_coll2.shard1.leader, tag=1245472810
   [junit4]   2> 551116 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://serv1-vm3:41607/dummyFolder/dataDir
   [junit4]   2> 551117 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://serv1-vm3:41607/dummyFolder/dataDir/index
   [junit4]   2> 551117 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://serv1-vm3:41607/dummyFolder/dataDir/snapshot_metadata
   [junit4]   2> 551117 ERROR (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error CREATEing SolrCore 'movereplicatest_coll2_shard1_replica_n0': Unable to create core [movereplicatest_coll2_shard1_replica_n0] Caused by: KeeperErrorCode = Session expired for /collections/movereplicatest_coll2
   [junit4]   2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:938)
   [junit4]   2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
   [junit4]   2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
   [junit4]   2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
   [junit4]   2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
   [junit4]   2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:745)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:726)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:507)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:378)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:322)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
   [junit4]   2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Unable to create core [movereplicatest_coll2_shard1_replica_n0]
   [junit4]   2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:985)
   [junit4]   2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:904)
   [junit4]   2> ... 34 more
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Failed to load config name for collection:movereplicatest_coll2 due to:
   [junit4]   2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:988)
   [junit4]   2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)
   [junit4]   2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:969)
   [junit4]   2> ... 35 more
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Failed to load config name for collection:movereplicatest_coll2 due to:
   [junit4]   2> at org.apache.solr.rest.ManagedResourceStorage.newStorageIO(ManagedResourceStorage.java:98)
   [junit4]   2> at org.apache.solr.core.SolrCore.initRestManager(SolrCore.java:2768)
   [junit4]   2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:963)
   [junit4]   2> ... 37 more
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Error loading config name for collection movereplicatest_coll2
   [junit4]   2> at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:206)
   [junit4]   2> at org.apache.solr.rest.ManagedResourceStorage.newStorageIO(ManagedResourceStorage.java:95)
   [junit4]   2> ... 39 more
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/movereplicatest_coll2
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
   [junit4]   2> at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:186)
   [junit4]   2> ... 40 more
   [junit4]   2>
   [junit4]   2> 551118 INFO  (qtp359362206-5851) [n:127.0.0.1:57615_solr c:movereplicatest_coll2 s:shard1 r:core_node1 x:movereplicatest_coll2_shard1_replica_n0] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node1&dataDir=hdfs://serv1-vm3:41607/dummyFolder/dataDir&collection.configName=conf1&name=movereplicatest_coll2_shard1_replica_n0&action=CREATE&collection=movereplicatest_coll2&shard=shard1&wt=javabin&version=2&ulogDir=hdfs://serv1-vm3:41607/dummyFolder2/ulogDir&replicaType=NRT} status=400 QTime=15295
   [junit4]   2> 551118 ERROR (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 551122 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33625 33625
   [junit4]   2> 551178 INFO  (Thread-1170) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33625 33625
   [junit4]   2> 551178 WARN  (Thread-1170) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/aliases.json
   [junit4]   2> 3 /solr/clusterprops.json
   [junit4]   2> 2 /solr/security.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/clusterstate.json
   [junit4]   2> 3 /solr/collections/movereplicatest_coll2/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/live_nodes
   [junit4]   2> 3 /solr/collections
   [junit4]   2>
   [junit4]   2> 551181 WARN  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
   [junit4]   2> 551193 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 551302 WARN  (DataNode: [[[DISK]file:/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSUlogDirTest_143E50FB78395A27-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSUlogDirTest_143E50FB78395A27-001/tempDir-002/hdfsBaseDir/data/data4/]]  heartbeating to serv1-vm3/127.0.0.1:41607) [    ] o.a.h.h.s.d.DataNode BPOfferService for Block pool BP-377772242-127.0.0.1-1500323250991 (Datanode Uuid dbe974f9-895f-4d18-9b56-35e6b31fe2b7) service to serv1-vm3/127.0.0.1:41607 interrupted
   [junit4]   2> 551302 WARN  (DataNode: [[[DISK]file:/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSUlogDirTest_143E50FB78395A27-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSUlogDirTest_143E50FB78395A27-001/tempDir-002/hdfsBaseDir/data/data4/]]  heartbeating to serv1-vm3/127.0.0.1:41607) [    ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-377772242-127.0.0.1-1500323250991 (Datanode Uuid dbe974f9-895f-4d18-9b56-35e6b31fe2b7) service to serv1-vm3/127.0.0.1:41607
   [junit4]   2> 551313 WARN  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
   [junit4]   2> 551331 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 551446 WARN  (DataNode: [[[DISK]file:/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSUlogDirTest_143E50FB78395A27-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSUlogDirTest_143E50FB78395A27-001/tempDir-002/hdfsBaseDir/data/data2/]]  heartbeating to serv1-vm3/127.0.0.1:41607) [    ] o.a.h.h.s.d.DataNode BPOfferService for Block pool BP-377772242-127.0.0.1-1500323250991 (Datanode Uuid c6b177a8-0655-461b-8cf7-1ae8ba85fabd) service to serv1-vm3/127.0.0.1:41607 interrupted
   [junit4]   2> 551447 WARN  (DataNode: [[[DISK]file:/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSUlogDirTest_143E50FB78395A27-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSUlogDirTest_143E50FB78395A27-001/tempDir-002/hdfsBaseDir/data/data2/]]  heartbeating to serv1-vm3/127.0.0.1:41607) [    ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-377772242-127.0.0.1-1500323250991 (Datanode Uuid c6b177a8-0655-461b-8cf7-1ae8ba85fabd) service to serv1-vm3/127.0.0.1:41607
   [junit4]   2> 551476 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@serv1-vm3:0
   [junit4]   2> 551587 ERROR (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.h.m.l.MethodMetric Error invoking method getBlocksTotal
   [junit4]   2> java.lang.reflect.InvocationTargetException
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2> at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
   [junit4]   2> at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
   [junit4]   2> at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:401)
   [junit4]   2> at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:194)
   [junit4]   2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:226)
   [junit4]   2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:211)
   [junit4]   2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:463)
   [junit4]   2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:213)
   [junit4]   2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:594)
   [junit4]   2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:171)
   [junit4]   2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
   [junit4]   2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
   [junit4]   2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
   [junit4]   2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:204)
   [junit4]   2> at org.apache.solr.cloud.MoveReplicaHDFSUlogDirTest.teardownClass(MoveReplicaHDFSUlogDirTest.java:66)
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
   [junit4]   2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
   [junit4]   2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
   [junit4]   2> ... 54 more
   [junit4]   2> 551592 INFO  (SUITE-MoveReplicaHDFSUlogDirTest-seed#[143E50FB78395A27]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33625 33625
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSUlogDirTest_143E50FB78395A27-001
   [junit4]   2> Jul 17, 2017 8:27:46 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 26 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=10236, maxDocsPerChunk=10, blockSize=248), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=10236, blockSize=248)), sim=RandomSimilarity(queryNorm=false): {}, locale=fr-BE, timezone=Hongkong
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=3,threads=2,free=82233640,total=373293056
   [junit4]   2> NOTE: All tests run in this JVM: [LeaderInitiatedRecoveryOnShardRestartTest, TestAtomicUpdateErrorCases, XsltUpdateRequestHandlerTest, NumericFieldsTest, CoreAdminHandlerTest, MetricUtilsTest, TestNRTOpen, TestCloudPseudoReturnFields, TestSchemalessBufferedUpdates, DisMaxRequestHandlerTest, TestSolrCloudSnapshots, TestCloudManagedSchema, BasicAuthIntegrationTest, RecoveryZkTest, SSLMigrationTest, TestHdfsBackupRestoreCore, BJQParserTest, SolrMetricManagerTest, SmileWriterTest, TestFieldCacheWithThreads, DataDrivenBlockJoinTest, CoreSorterTest, TestReqParamsAPI, HdfsTlogReplayBufferedWhileIndexingTest, TestSizeLimitedDistributedMap, TestBlobHandler, TestDeleteCollectionOnDownNodes, RequiredFieldsTest, TestSolrConfigHandlerCloud, TestConfigSetsAPI, TestMissingGroups, TestOrdValues, DateMathParserTest, TestExactStatsCache, XmlUpdateRequestHandlerTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestMacros, TestApiFramework, TestSort, HdfsNNFailoverTest, TestLegacyNumericUtils, BlobRepositoryCloudTest, TestUpdate, TestOmitPositions, TestNumericRangeQuery64, MissingSegmentRecoveryTest, SuggesterWFSTTest, EnumFieldTest, ClusterStateTest, TestInPlaceUpdatesDistrib, TestCustomSort, CloneFieldUpdateProcessorFactoryTest, TestLegacyField, TestEmbeddedSolrServerSchemaAPI, TestHighFrequencyDictionaryFactory, MoveReplicaHDFSUlogDirTest]
   [junit4] Completed [114/728 (1!)] on J0 in 25.80s, 1 test, 1 error <<< FAILURES!

[...truncated 45415 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 49 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/49/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 6 object(s) that were not released!!! [MockDirectoryWrapper, InternalHttpClient, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)  at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:92)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:741)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:934)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:969)  at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:610)  at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:298)  at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:220)  at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:254)  at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1213)  at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:696)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:967)  at org.apache.solr.core.SolrCore.reload(SolrCore.java:636)  at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1202)  at org.apache.solr.handler.IndexFetcher.lambda$reloadCore$0(IndexFetcher.java:900)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)  at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:478)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:928)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:969)  at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:610)  at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)  at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:349)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:709)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:934)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:969)  at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:610)  at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1019)  at org.apache.solr.core.SolrCore.reload(SolrCore.java:636)  at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1202)  at org.apache.solr.handler.IndexFetcher.lambda$reloadCore$0(IndexFetcher.java:900)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)  at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:484)  at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332)  at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)  at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1183)  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  

Stack Trace:
java.lang.AssertionError: ObjectTracker found 6 object(s) that were not released!!! [MockDirectoryWrapper, InternalHttpClient, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
        at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
        at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:92)
        at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:741)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:934)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:969)
        at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:610)
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
        at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)
        at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:298)
        at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:220)
        at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:254)
        at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1213)
        at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:696)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:967)
        at org.apache.solr.core.SolrCore.reload(SolrCore.java:636)
        at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1202)
        at org.apache.solr.handler.IndexFetcher.lambda$reloadCore$0(IndexFetcher.java:900)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
        at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
        at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:478)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:928)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:969)
        at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:610)
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
        at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
        at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:349)
        at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:709)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:934)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:969)
        at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:610)
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
        at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1019)
        at org.apache.solr.core.SolrCore.reload(SolrCore.java:636)
        at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1202)
        at org.apache.solr.handler.IndexFetcher.lambda$reloadCore$0(IndexFetcher.java:900)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
        at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
        at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:484)
        at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332)
        at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)
        at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1183)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)


        at __randomizedtesting.SeedInfo.seed([80ED5FC3E37F83EC]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:296)
        at sun.reflect.GeneratedMethodAccessor47.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12095 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/init-core-data-001
   [junit4]   2> 1468062 WARN  (SUITE-TestReplicationHandler-seed#[80ED5FC3E37F83EC]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=11 numCloses=11
   [junit4]   2> 1468062 INFO  (SUITE-TestReplicationHandler-seed#[80ED5FC3E37F83EC]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1468063 INFO  (SUITE-TestReplicationHandler-seed#[80ED5FC3E37F83EC]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
   [junit4]   2> 1468065 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.SolrTestCaseJ4 ###Starting doTestIndexAndConfigAliasReplication
   [junit4]   2> 1468066 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/solr-instance-001/collection1
   [junit4]   2> 1468069 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1468070 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@469abf7b{/solr,null,AVAILABLE}
   [junit4]   2> 1468070 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@61dd5c14{HTTP/1.1,[http/1.1]}{127.0.0.1:62487}
   [junit4]   2> 1468071 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.e.j.s.Server Started @1470917ms
   [junit4]   2> 1468071 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=62487}
   [junit4]   2> 1468071 ERROR (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1468071 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.1.0
   [junit4]   2> 1468071 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 1468071 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1468071 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-18T04:02:18.684Z
   [junit4]   2> 1468072 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/solr-instance-001/solr.xml
   [junit4]   2> 1468077 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2579773d, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1468105 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2579773d
   [junit4]   2> 1468109 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2579773d
   [junit4]   2> 1468109 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2579773d
   [junit4]   2> 1468110 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/solr-instance-001/.
   [junit4]   2> 1468110 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 1468119 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 1468123 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1468124 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
   [junit4]   2> 1468124 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/solr-instance-001/./collection1, trusted=true
   [junit4]   2> 1468125 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2579773d
   [junit4]   2> 1468125 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1468125 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/solr-instance-001/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 1468126 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=57.7001953125, floorSegmentMB=0.3271484375, forceMergeDeletesPctAllowed=22.375226668229615, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1468162 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1468162 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1468162 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=44, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=21.5166015625, floorSegmentMB=0.9365234375, forceMergeDeletesPctAllowed=0.15721322126448722, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1468163 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@378ec3f2[collection1] main]
   [junit4]   2> 1468163 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/solr-instance-001/collection1/conf
   [junit4]   2> 1468163 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 1468163 INFO  (coreLoadExecutor-5669-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1468164 INFO  (searcherExecutor-5670-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@378ec3f2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1468623 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/solr-instance-002/collection1
   [junit4]   2> 1468644 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1468646 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@69654506{/solr,null,AVAILABLE}
   [junit4]   2> 1468647 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@232865de{HTTP/1.1,[http/1.1]}{127.0.0.1:60269}
   [junit4]   2> 1468647 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.e.j.s.Server Started @1471493ms
   [junit4]   2> 1468647 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=60269}
   [junit4]   2> 1468647 ERROR (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1468647 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.1.0
   [junit4]   2> 1468648 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 1468648 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1468648 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-18T04:02:19.261Z
   [junit4]   2> 1468649 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/solr-instance-002/solr.xml
   [junit4]   2> 1468656 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2579773d, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1468717 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2579773d
   [junit4]   2> 1468724 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2579773d
   [junit4]   2> 1468724 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2579773d
   [junit4]   2> 1468726 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/solr-instance-002/.
   [junit4]   2> 1468726 INFO  (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[80ED5FC3E37F83EC]) [    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 1468744 INFO  (coreLoadExecutor-5679-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 1468750 INFO  (coreLoadExecutor-5679-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1468752 INFO  (coreLoadExecutor-5679-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
   [junit4]   2> 1468752 INFO  (coreLoadExecutor-5679-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/solr-instance-002/./collection1, trusted=true
   [junit4]   2> 1468752 INFO  (coreLoadExecutor-5679-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2579773d
   [junit4]   2> 1468752 INFO  (coreLoadExecutor-5679-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1468752 INFO  (coreLoadExecutor-5679-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/solr-instance-002/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/solr-instance-002/./collection1/data/]
   [junit4]   2> 1468754 INFO  (coreLoadExecutor-5679-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=57.7001953125, floorSegmentMB=0.3271484375, forceMergeDeletesPctAllowed=22.375226668229615, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1468794 INFO  (coreLoadExecutor-5679-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1468794 INFO  (coreLoadExecutor-5679-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1468794 INFO  (coreLoadExecutor-5679-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=44, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=21.5166015625, floorSegmentMB=0.9365234375, forceMergeDeletesPctAllowed=0.15721322126448722, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1468794 INFO  (coreLoadExecutor-5679-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@76a421db[collection1] main]
   [junit4]   2> 1468795 INFO  (coreLoadExecutor-5679-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_80ED5FC3E37F83EC-001/solr-instance-002/collection1/conf
   [junit4]   2> 1468796 INFO  (coreLoadExecutor-5679-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 1468798 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 1468798 INFO  (searcherExecutor-5680-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@76a421db[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1468798 INFO  (indexFetcher-5684-thread-1) [    x:collection1] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 1468798 INFO  (indexFetcher-5684-thread-1) [    x:collection1] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 1468798 INFO  (indexFetcher-5684-thread-1) [    x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 1468798 INFO  (indexFetcher-5684-thread-1) [    x:collection1] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 1469261 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/select params={q=*:*&sort=id+desc&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1469262 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 0
   [junit4]   2> 1469263 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 0
   [junit4]   2> 1469264 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
   [junit4]   2> 1469265 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[3]} 0 0
   [junit4]   2> 1469266 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[4]} 0 0
   [junit4]   2> 1469267 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[5]} 0 0
   [junit4]   2> 1469267 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[6]} 0 0
   [junit4]   2> 1469268 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[7]} 0 0
   [junit4]   2> 1469269 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[8]} 0 0
   [junit4]   2> 1469270 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[9]} 0 0
   [junit4]   2> 1469271 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[10]} 0 0
   [junit4]   2> 1469271 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[11]} 0 0
   [junit4]   2> 1469272 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[12]} 0 0
   [junit4]   2> 1469273 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[13]} 0 0
   [junit4]   2> 1469274 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[14]} 0 0
   [junit4]   2> 1469275 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[15]} 0 0
   [junit4]   2> 1469276 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[16]} 0 0
   [junit4]   2> 1469276 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[17]} 0 0
   [junit4]   2> 1469277 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[18]} 0 0
   [junit4]   2> 1469278 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[19]} 0 0
   [junit4]   2> 1469279 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[20]} 0 0
   [junit4]   2> 1469280 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[21]} 0 0
   [junit4]   2> 1469281 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[22]} 0 0
   [junit4]   2> 1469281 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[23]} 0 0
   [junit4]   2> 1469282 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[24]} 0 0
   [junit4]   2> 1469283 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[25]} 0 0
   [junit4]   2> 1469284 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[26]} 0 0
   [junit4]   2> 1469285 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[27]} 0 0
   [junit4]   2> 1469286 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[28]} 0 0
   [junit4]   2> 1469287 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[29]} 0 0
   [junit4]   2> 1469288 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[30]} 0 0
   [junit4]   2> 1469288 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[31]} 0 0
   [junit4]   2> 1469289 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[32]} 0 0
   [junit4]   2> 1469289 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[33]} 0 0
   [junit4]   2> 1469290 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[34]} 0 0
   [junit4]   2> 1469291 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[35]} 0 0
   [junit4]   2> 1469291 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[36]} 0 0
   [junit4]   2> 1469292 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[37]} 0 0
   [junit4]   2> 1469292 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[38]} 0 0
   [junit4]   2> 1469292 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[39]} 0 0
   [junit4]   2> 1469293 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[40]} 0 0
   [junit4]   2> 1469293 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[41]} 0 0
   [junit4]   2> 1469294 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[42]} 0 0
   [junit4]   2> 1469294 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[43]} 0 0
   [junit4]   2> 1469294 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[44]} 0 0
   [junit4]   2> 1469295 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[45]} 0 0
   [junit4]   2> 1469295 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[46]} 0 0
   [junit4]   2> 1469296 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[47]} 0 0
   [junit4]   2> 1469296 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[48]} 0 0
   [junit4]   2> 1469297 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[49]} 0 0
   [junit4]   2> 1469297 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[50]} 0 0
   [junit4]   2> 1469297 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[51]} 0 0
   [junit4]   2> 1469298 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[52]} 0 0
   [junit4]   2> 1469299 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[53]} 0 0
   [junit4]   2> 1469299 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[54]} 0 0
   [junit4]   2> 1469300 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[55]} 0 0
   [junit4]   2> 1469300 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[56]} 0 0
   [junit4]   2> 1469301 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[57]} 0 0
   [junit4]   2> 1469301 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[58]} 0 0
   [junit4]   2> 1469302 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[59]} 0 0
   [junit4]   2> 1469303 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[60]} 0 0
   [junit4]   2> 1469309 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[61]} 0 2
   [junit4]   2> 1469310 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[62]} 0 0
   [junit4]   2> 1469313 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[63]} 0 0
   [junit4]   2> 1469314 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[64]} 0 0
   [junit4]   2> 1469314 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[65]} 0 0
   [junit4]   2> 1469315 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[66]} 0 0
   [junit4]   2> 1469315 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[67]} 0 0
   [junit4]   2> 1469315 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[68]} 0 0
   [junit4]   2> 1469315 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[69]} 0 0
   [junit4]   2> 1469316 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[70]} 0 0
   [junit4]   2> 1469316 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[71]} 0 0
   [junit4]   2> 1469316 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[72]} 0 0
   [junit4]   2> 1469316 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[73]} 0 0
   [junit4]   2> 1469316 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[74]} 0 0
   [junit4]   2> 1469317 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[75]} 0 0
   [junit4]   2> 1469317 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[76]} 0 0
   [junit4]   2> 1469317 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[77]} 0 0
   [junit4]   2> 1469317 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[78]} 0 0
   [junit4]   2> 1469317 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[79]} 0 0
   [junit4]   2> 1469318 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[80]} 0 0
   [junit4]   2> 1469318 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[81]} 0 0
   [junit4]   2> 1469318 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[82]} 0 0
   [junit4]   2> 1469318 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[83]} 0 0
   [junit4]   2> 1469318 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[84]} 0 0
   [junit4]   2> 1469319 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[85]} 0 0
   [junit4]   2> 1469319 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[86]} 0 0
   [junit4]   2> 1469319 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[87]} 0 0
   [junit4]   2> 1469319 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[88]} 0 0
   [junit4]   2> 1469319 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[89]} 0 0
   [junit4]   2> 1469320 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[90]} 0 0
   [junit4]   2> 1469320 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[91]} 0 0
   [junit4]   2> 1469320 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[92]} 0 0
   [junit4]   2> 1469320 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[93]} 0 0
   [junit4]   2> 1469320 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[94]} 0 0
   [junit4]   2> 1469321 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[95]} 0 0
   [junit4]   2> 1469321 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[96]} 0 0
   [junit4]   2> 1469321 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[97]} 0 0
   [junit4]   2> 1469321 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[98]} 0 0
   [junit4]   2> 1469322 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[99]} 0 0
   [junit4]   2> 1469322 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[100]} 0 0
   [junit4]   2> 1469322 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[101]} 0 0
   [junit4]   2> 1469322 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[102]} 0 0
   [junit4]   2> 1469322 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[103]} 0 0
   [junit4]   2> 1469322 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[104]} 0 0
   [junit4]   2> 1469323 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[105]} 0 0
   [junit4]   2> 1469323 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[106]} 0 0
   [junit4]   2> 1469324 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[107]} 0 0
   [junit4]   2> 1469327 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[108]} 0 1
   [junit4]   2> 1469328 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[109]} 0 0
   [junit4]   2> 1469328 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[110]} 0 0
   [junit4]   2> 1469328 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[111]} 0 0
   [junit4]   2> 1469329 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[112]} 0 0
   [junit4]   2> 1469329 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[113]} 0 0
   [junit4]   2> 1469329 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[114]} 0 0
   [junit4]   2> 1469330 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[115]} 0 0
   [junit4]   2> 1469330 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[116]} 0 0
   [junit4]   2> 1469330 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[117]} 0 0
   [junit4]   2> 1469331 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[118]} 0 0
   [junit4]   2> 1469331 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[119]} 0 0
   [junit4]   2> 1469331 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[120]} 0 0
   [junit4]   2> 1469332 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[121]} 0 0
   [junit4]   2> 1469332 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[122]} 0 0
   [junit4]   2> 1469332 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[123]} 0 0
   [junit4]   2> 1469333 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[124]} 0 0
   [junit4]   2> 1469333 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[125]} 0 0
   [junit4]   2> 1469334 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[126]} 0 0
   [junit4]   2> 1469334 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[127]} 0 0
   [junit4]   2> 1469334 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[128]} 0 0
   [junit4]   2> 1469335 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[129]} 0 0
   [junit4]   2> 1469335 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[130]} 0 0
   [junit4]   2> 1469335 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[131]} 0 0
   [junit4]   2> 1469336 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[132]} 0 0
   [junit4]   2> 1469336 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[133]} 0 0
   [junit4]   2> 1469337 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[134]} 0 0
   [junit4]   2> 1469337 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[135]} 0 0
   [junit4]   2> 1469337 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[136]} 0 0
   [junit4]   2> 1469338 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[137]} 0 0
   [junit4]   2> 1469338 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[138]} 0 0
   [junit4]   2> 1469339 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[139]} 0 0
   [junit4]   2> 1469339 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[140]} 0 0
   [junit4]   2> 1469339 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[141]} 0 0
   [junit4]   2> 1469340 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[142]} 0 0
   [junit4]   2> 1469340 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[143]} 0 0
   [junit4]   2> 1469340 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[144]} 0 0
   [junit4]   2> 1469341 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[145]} 0 0
   [junit4]   2> 1469341 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[146]} 0 0
   [junit4]   2> 1469341 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[147]} 0 0
   [junit4]   2> 1469342 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[148]} 0 0
   [junit4]   2> 1469342 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[149]} 0 0
   [junit4]   2> 1469342 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[150]} 0 0
   [junit4]   2> 1469343 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[151]} 0 0
   [junit4]   2> 1469343 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[152]} 0 0
   [junit4]   2> 1469343 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[153]} 0 0
   [junit4]   2> 1469344 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[154]} 0 0
   [junit4]   2> 1469344 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[155]} 0 0
   [junit4]   2> 1469345 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[156]} 0 0
   [junit4]   2> 1469345 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[157]} 0 0
   [junit4]   2> 1469345 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[158]} 0 0
   [junit4]   2> 1469346 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[159]} 0 0
   [junit4]   2> 1469346 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[160]} 0 0
   [junit4]   2> 1469346 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[161]} 0 0
   [junit4]   2> 1469347 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[162]} 0 0
   [junit4]   2> 1469347 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[163]} 0 0
   [junit4]   2> 1469348 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[164]} 0 0
   [junit4]   2> 1469348 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[165]} 0 0
   [junit4]   2> 1469348 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[166]} 0 0
   [junit4]   2> 1469349 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[167]} 0 0
   [junit4]   2> 1469349 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[168]} 0 0
   [junit4]   2> 1469349 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[169]} 0 0
   [junit4]   2> 1469350 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[170]} 0 0
   [junit4]   2> 1469350 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[171]} 0 0
   [junit4]   2> 1469350 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[172]} 0 0
   [junit4]   2> 1469351 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[173]} 0 0
   [junit4]   2> 1469351 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[174]} 0 0
   [junit4]   2> 1469351 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[175]} 0 0
   [junit4]   2> 1469352 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[176]} 0 0
   [junit4]   2> 1469352 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[177]} 0 0
   [junit4]   2> 1469352 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[178]} 0 0
   [junit4]   2> 1469353 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[179]} 0 0
   [junit4]   2> 1469353 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[180]} 0 0
   [junit4]   2> 1469354 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[181]} 0 0
   [junit4]   2> 1469354 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[182]} 0 0
   [junit4]   2> 1469354 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[183]} 0 0
   [junit4]   2> 1469355 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[184]} 0 0
   [junit4]   2> 1469355 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[185]} 0 0
   [junit4]   2> 1469355 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[186]} 0 0
   [junit4]   2> 1469356 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[187]} 0 0
   [junit4]   2> 1469356 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[188]} 0 0
   [junit4]   2> 1469356 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[189]} 0 0
   [junit4]   2> 1469357 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[190]} 0 0
   [junit4]   2> 1469357 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[191]} 0 0
   [junit4]   2> 1469358 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[192]} 0 0
   [junit4]   2> 1469358 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[193]} 0 0
   [junit4]   2> 1469358 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[194]} 0 0
   [junit4]   2> 1469359 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[195]} 0 0
   [junit4]   2> 1469359 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[196]} 0 0
   [junit4]   2> 1469360 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[197]} 0 0
   [junit4]   2> 1469360 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[198]} 0 0
   [junit4]   2> 1469360 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[199]} 0 0
   [junit4]   2> 1469361 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[200]} 0 0
   [junit4]   2> 1469361 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[201]} 0 0
   [junit4]   2> 1469361 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[202]} 0 0
   [junit4]   2> 1469362 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[203]} 0 0
   [junit4]   2> 1469362 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[204]} 0 0
   [junit4]   2> 1469362 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[205]} 0 0
   [junit4]   2> 1469363 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[206]} 0 0
   [junit4]   2> 1469363 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[207]} 0 0
   [junit4]   2> 1469364 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[208]} 0 0
   [junit4]   2> 1469364 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[209]} 0 0
   [junit4]   2> 1469364 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[210]} 0 0
   [junit4]   2> 1469365 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[211]} 0 0
   [junit4]   2> 1469365 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[212]} 0 0
   [junit4]   2> 1469365 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[213]} 0 0
   [junit4]   2> 1469366 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[214]} 0 0
   [junit4]   2> 1469366 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[215]} 0 0
   [junit4]   2> 1469366 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[216]} 0 0
   [junit4]   2> 1469367 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[217]} 0 0
   [junit4]   2> 1469367 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[218]} 0 0
   [junit4]   2> 1469367 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[219]} 0 0
   [junit4]   2> 1469368 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[220]} 0 0
   [junit4]   2> 1469368 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[221]} 0 0
   [junit4]   2> 1469369 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[222]} 0 0
   [junit4]   2> 1469369 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[223]} 0 0
   [junit4]   2> 1469369 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[224]} 0 0
   [junit4]   2> 1469370 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[225]} 0 0
   [junit4]   2> 1469370 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[226]} 0 0
   [junit4]   2> 1469370 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[227]} 0 0
   [junit4]   2> 1469371 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[228]} 0 0
   [junit4]   2> 1469371 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[229]} 0 0
   [junit4]   2> 1469371 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[230]} 0 0
   [junit4]   2> 1469372 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[231]} 0 0
   [junit4]   2> 1469372 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[232]} 0 0
   [junit4]   2> 1469373 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[233]} 0 0
   [junit4]   2> 1469373 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[234]} 0 0
   [junit4]   2> 1469373 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[235]} 0 0
   [junit4]   2> 1469374 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[236]} 0 0
   [junit4]   2> 1469374 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[237]} 0 0
   [junit4]   2> 1469374 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[238]} 0 0
   [junit4]   2> 1469375 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[239]} 0 0
   [junit4]   2> 1469375 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[240]} 0 0
   [junit4]   2> 1469375 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[241]} 0 0
   [junit4]   2> 1469376 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[242]} 0 0
   [junit4]   2> 1469376 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[243]} 0 0
   [junit4]   2> 1469377 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[244]} 0 0
   [junit4]   2> 1469377 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[245]} 0 0
   [junit4]   2> 1469377 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[246]} 0 0
   [junit4]   2> 1469378 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[247]} 0 0
   [junit4]   2> 1469378 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[248]} 0 0
   [junit4]   2> 1469378 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[249]} 0 0
   [junit4]   2> 1469379 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[250]} 0 0
   [junit4]   2> 1469379 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[251]} 0 0
   [junit4]   2> 1469379 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[252]} 0 0
   [junit4]   2> 1469380 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[253]} 0 0
   [junit4]   2> 1469380 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[254]} 0 0
   [junit4]   2> 1469380 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[255]} 0 0
   [junit4]   2> 1469381 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[256]} 0 0
   [junit4]   2> 1469381 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[257]} 0 0
   [junit4]   2> 1469381 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[258]} 0 0
   [junit4]   2> 1469381 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[259]} 0 0
   [junit4]   2> 1469382 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[260]} 0 0
   [junit4]   2> 1469382 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[261]} 0 0
   [junit4]   2> 1469383 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[262]} 0 0
   [junit4]   2> 1469383 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[263]} 0 0
   [junit4]   2> 1469383 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[264]} 0 0
   [junit4]   2> 1469384 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[265]} 0 0
   [junit4]   2> 1469384 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[266]} 0 0
   [junit4]   2> 1469384 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[267]} 0 0
   [junit4]   2> 1469385 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[268]} 0 0
   [junit4]   2> 1469385 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[269]} 0 0
   [junit4]   2> 1469386 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[270]} 0 0
   [junit4]   2> 1469386 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[271]} 0 0
   [junit4]   2> 1469386 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[272]} 0 0
   [junit4]   2> 1469387 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[273]} 0 0
   [junit4]   2> 1469387 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[274]} 0 0
   [junit4]   2> 1469387 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[275]} 0 0
   [junit4]   2> 1469388 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[276]} 0 0
   [junit4]   2> 1469388 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[277]} 0 0
   [junit4]   2> 1469388 INFO  (qtp1904931773-16587) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[278]} 0 0
   [junit4]   2> 1469389 INFO  (qtp1904931773-16586) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[279]} 0 0
   [junit4]   2> 1469389 INFO  (qtp1904931773-16590) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[280]} 0 0
   [junit4]   2> 1469390 INFO  (qtp1904931773-16592) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[281]} 0 0
   [junit4]   2> 1469390 INFO  (qtp1904931773-16589) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[282]} 0 0
   [junit4]   2> 1469390 INFO  (qtp1904931773-16588) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[283]} 0 0
   [junit4]   2> 1469390 INFO  (qtp1904931773-16593) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFacto

[...truncated too long message...]

oaderTest, TestWriterPerf, TestStressCloudBlindAtomicUpdates, TestPhraseSuggestions, PKIAuthenticationIntegrationTest, TestClusterStateMutator, TestShardHandlerFactory, SmileWriterTest, TestObjectReleaseTracker, BasicDistributedZkTest, BlockDirectoryTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestComplexPhraseLeadingWildcard, SortByFunctionTest, TestInfoStreamLogging, TestLegacyField, ExitableDirectoryReaderTest, TestBlobHandler, TimeZoneUtilsTest, TestLegacyNumericUtils, ChaosMonkeyNothingIsSafeTest, AtomicUpdatesTest, MultiThreadedOCPTest, HdfsBasicDistributedZkTest, LeaderElectionContextKeyTest, TestFuzzyAnalyzedSuggestions, TestTolerantSearch, SSLMigrationTest, DistributedFacetPivotSmallTest, TestUninvertingReader, AnalyticsQueryTest, HdfsSyncSliceTest, LeaderInitiatedRecoveryOnShardRestartTest, TestOmitPositions, AsyncCallRequestStatusResponseTest, TestMultiValuedNumericRangeQuery, TestStandardQParsers, OutputWriterTest, NoCacheHeaderTest, PropertiesRequestHandlerTest, TestIndexingPerformance, TestCustomSort, BJQParserTest, TestSweetSpotSimilarityFactory, TestHashQParserPlugin, DistribJoinFromCollectionTest, RestartWhileUpdatingTest, AnalysisErrorHandlingTest, CursorPagingTest, TestTolerantUpdateProcessorCloud, SolrInfoBeanTest, QueryParsingTest, DistributedVersionInfoTest, HdfsTlogReplayBufferedWhileIndexingTest, DistributedSuggestComponentTest, CollectionsAPISolrJTest, TestHttpShardHandlerFactory, TestXmlQParserPlugin, TestReplicationHandler]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestReplicationHandler -Dtests.seed=80ED5FC3E37F83EC -Dtests.slow=true -Dtests.locale=no -Dtests.timezone=America/Atikokan -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J0 | TestReplicationHandler (suite) <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: ObjectTracker found 6 object(s) that were not released!!! [MockDirectoryWrapper, InternalHttpClient, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper]
   [junit4]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
   [junit4]    > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]    > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
   [junit4]    > at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:92)
   [junit4]    > at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:741)
   [junit4]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:934)
   [junit4]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)
   [junit4]    > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:969)
   [junit4]    > at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:610)
   [junit4]    > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
   [junit4]    > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]    > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]    > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]    > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
   [junit4]    > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:298)
   [junit4]    > at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:220)
   [junit4]    > at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:254)
   [junit4]    > at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1213)
   [junit4]    > at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:696)
   [junit4]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:967)
   [junit4]    > at org.apache.solr.core.SolrCore.reload(SolrCore.java:636)
   [junit4]    > at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1202)
   [junit4]    > at org.apache.solr.handler.IndexFetcher.lambda$reloadCore$0(IndexFetcher.java:900)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
   [junit4]    > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]    > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
   [junit4]    > at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:478)
   [junit4]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:928)
   [junit4]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)
   [junit4]    > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:969)
   [junit4]    > at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:610)
   [junit4]    > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
   [junit4]    > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]    > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]    > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]    > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
   [junit4]    > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]    > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
   [junit4]    > at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:349)
   [junit4]    > at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:709)
   [junit4]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:934)
   [junit4]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)
   [junit4]    > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:969)
   [junit4]    > at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:610)
   [junit4]    > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
   [junit4]    > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]    > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]    > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]    > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
   [junit4]    > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1019)
   [junit4]    > at org.apache.solr.core.SolrCore.reload(SolrCore.java:636)
   [junit4]    > at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1202)
   [junit4]    > at org.apache.solr.handler.IndexFetcher.lambda$reloadCore$0(IndexFetcher.java:900)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
   [junit4]    > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]    > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
   [junit4]    > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:484)
   [junit4]    > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332)
   [junit4]    > at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)
   [junit4]    > at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1183)
   [junit4]    > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]    > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
   [junit4]    > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
   [junit4]    > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
   [junit4]    > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]    > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([80ED5FC3E37F83EC]:0)
   [junit4]    > at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:296)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4] Completed [382/728 (1!)] on J0 in 104.05s, 16 tests, 1 failure, 1 skipped <<< FAILURES!

[...truncated 44416 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 50 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/50/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.update.AutoCommitTest.testCommitWithin

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
        at __randomizedtesting.SeedInfo.seed([570BC2F3989CE34B:EDD9AD8B1BB20D5E]:0)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:878)
        at org.apache.solr.update.AutoCommitTest.testCommitWithin(AutoCommitTest.java:353)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
        xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>

        request was:q=id:529&qt=&start=0&rows=20&version=2.2
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:871)
        ... 40 more




Build Log:
[...truncated 12066 lines...]
   [junit4] Suite: org.apache.solr.update.AutoCommitTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.update.AutoCommitTest_570BC2F3989CE34B-001/init-core-data-001
   [junit4]   2> 1881171 WARN  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=309 numCloses=309
   [junit4]   2> 1881171 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1881172 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 1881172 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 1881172 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/lib, /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1881191 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 1881205 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 1881272 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1881300 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41838b98
   [junit4]   2> 1881302 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41838b98
   [junit4]   2> 1881302 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41838b98
   [junit4]   2> 1881303 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/lib, /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1881322 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 1881335 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1881397 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1881405 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 1881405 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41838b98
   [junit4]   2> 1881405 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1881405 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.update.AutoCommitTest_570BC2F3989CE34B-001/init-core-data-001/]
   [junit4]   2> 1881407 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=50, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8468563289541424]
   [junit4]   2> 1881461 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1881461 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1881462 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1881462 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1881462 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=49, maxMergedSegmentMB=45.1669921875, floorSegmentMB=1.841796875, forceMergeDeletesPctAllowed=13.664139187493822, segmentsPerTier=31.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1881463 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@31f7acc5[collection1] main]
   [junit4]   2> 1881463 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1881466 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1881478 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1881478 INFO  (searcherExecutor-7838-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1881478 INFO  (coreLoadExecutor-7837-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1573259220322615296
   [junit4]   2> 1881479 INFO  (searcherExecutor-7838-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 1881479 INFO  (searcherExecutor-7838-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1881479 INFO  (searcherExecutor-7838-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1881479 INFO  (searcherExecutor-7838-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1881479 INFO  (searcherExecutor-7838-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 1881479 INFO  (searcherExecutor-7838-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 1881480 INFO  (searcherExecutor-7838-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 1881480 INFO  (searcherExecutor-7838-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 1881481 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 1881481 INFO  (searcherExecutor-7838-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@31f7acc5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1881483 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMaxDocs
   [junit4]   2> 1881484 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 1881484 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@17e420df[collection1] realtime]
   [junit4]   2> 1881484 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 0
   [junit4]   2> 1881484 ERROR (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    ] o.a.s.c.CorePropertiesLocator Couldn't load core descriptor from /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/core.properties:java.nio.file.NoSuchFileException: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/core.properties
   [junit4]   2> 1881485 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/lib, /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1881506 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 1881520 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1881608 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1881614 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1
   [junit4]   2> 1881624 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41838b98
   [junit4]   2> 1881624 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.update.AutoCommitTest_570BC2F3989CE34B-001/init-core-data-001/]
   [junit4]   2> 1881688 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1881688 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1881689 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@402e4e97[collection1] main]
   [junit4]   2> 1881690 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1881690 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1881692 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1881692 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1573259220547010560
   [junit4]   2> 1881695 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=12, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6958451202820501]
   [junit4]   2> 1881695 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1881695 INFO  (searcherExecutor-7843-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1881696 INFO  (searcherExecutor-7843-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 1881696 INFO  (searcherExecutor-7843-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1881696 INFO  (searcherExecutor-7843-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1881696 INFO  (searcherExecutor-7843-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1881696 INFO  (searcherExecutor-7843-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 1881696 INFO  (searcherExecutor-7843-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 1881697 INFO  (searcherExecutor-7843-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 1881697 INFO  (searcherExecutor-7843-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 1881698 INFO  (searcherExecutor-7843-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@402e4e97[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1881698 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@59d25a54[collection1] main]
   [junit4]   2> 1881699 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@70a8996
   [junit4]   2> 1881699 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=118131094
   [junit4]   2> 1881706 INFO  (searcherExecutor-7843-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@59d25a54[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1881707 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[0 (1573259220560642048)]} 0 2
   [junit4]   2> 1881707 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[1 (1573259220562739200)]} 0 2
   [junit4]   2> 1881707 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[2 (1573259220562739201)]} 0 2
   [junit4]   2> 1881707 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[3 (1573259220562739202)]} 0 2
   [junit4]   2> 1881708 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[4 (1573259220563787776)]} 0 3
   [junit4]   2> 1881708 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[5 (1573259220563787777)]} 0 3
   [junit4]   2> 1881708 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[6 (1573259220563787778)]} 0 3
   [junit4]   2> 1881708 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[7 (1573259220563787779)]} 0 3
   [junit4]   2> 1881708 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[8 (1573259220563787780)]} 0 3
   [junit4]   2> 1881708 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[9 (1573259220563787781)]} 0 3
   [junit4]   2> 1881708 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[10 (1573259220563787782)]} 0 3
   [junit4]   2> 1881709 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[11 (1573259220564836352)]} 0 4
   [junit4]   2> 1881709 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[12 (1573259220564836353)]} 0 4
   [junit4]   2> 1881709 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[13 (1573259220564836354)]} 0 4
   [junit4]   2> 1881709 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:1&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1881711 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[14 (1573259220566933504)]} 0 6
   [junit4]   2> 1881713 INFO  (commitScheduler-7846-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
   [junit4]   2> 1881723 INFO  (commitScheduler-7846-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@33f441d3[collection1] main]
   [junit4]   2> 1881725 INFO  (commitScheduler-7846-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1881731 INFO  (searcherExecutor-7843-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@33f441d3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):c15)))}
   [junit4]   2> 1881970 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[15 (1573259220838514688)]} 0 265
   [junit4]   2> 1881970 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:14&qt=&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
   [junit4]   2> 1881971 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:15&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1881971 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testMaxDocs
   [junit4]   2> 1881976 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Starting testCommitWithin
   [junit4]   2> 1881977 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 1881977 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@64abbebb[collection1] realtime]
   [junit4]   2> 1881977 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 0
   [junit4]   2> 1881977 ERROR (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.CorePropertiesLocator Couldn't load core descriptor from /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/core.properties:java.nio.file.NoSuchFileException: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/core.properties
   [junit4]   2> 1881978 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/lib, /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1882031 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 1882062 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1882189 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1882236 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1
   [junit4]   2> 1882249 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41838b98
   [junit4]   2> 1882249 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.update.AutoCommitTest_570BC2F3989CE34B-001/init-core-data-001/]
   [junit4]   2> 1882353 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1882353 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1882355 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2a38142d[collection1] main]
   [junit4]   2> 1882356 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1882356 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1882363 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1882363 INFO  (searcherExecutor-7847-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1882364 INFO  (searcherExecutor-7847-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 1882366 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=41, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1882366 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1882366 INFO  (searcherExecutor-7847-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1882366 INFO  (searcherExecutor-7847-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1882367 INFO  (searcherExecutor-7847-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1882368 INFO  (searcherExecutor-7847-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 1882368 INFO  (searcherExecutor-7847-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 1882368 INFO  (searcherExecutor-7847-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 1882368 INFO  (searcherExecutor-7847-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 1882370 INFO  (searcherExecutor-7847-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2a38142d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1882370 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@263c4d07[collection1] main]
   [junit4]   2> 1882370 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@32671e0b
   [junit4]   2> 1882370 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=845618699
   [junit4]   2> 1882373 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[529 (1573259221260042240)]} 0 0
   [junit4]   2> 1882373 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1882375 INFO  (searcherExecutor-7847-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@263c4d07[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1882624 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[530 (1573259221524283392)]} 0 252
   [junit4]   2> 1882625 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:530&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1882625 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[529 (-1573259221525331968)]} 0 253
   [junit4]   2> 1882625 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1882626 ERROR (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2>
   [junit4]   2> request was:q=id:529&qt=&start=0&rows=20&version=2.2
   [junit4]   2> 1882626 ERROR (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: q=id:529&qt=&start=0&rows=20&version=2.2:java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2>
   [junit4]   2> request was:q=id:529&qt=&start=0&rows=20&version=2.2
   [junit4]   2> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:871)
   [junit4]   2> at org.apache.solr.update.AutoCommitTest.testCommitWithin(AutoCommitTest.java:353)
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>
   [junit4]   2> 1882626 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testCommitWithin
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=AutoCommitTest -Dtests.method=testCommitWithin -Dtests.seed=570BC2F3989CE34B -Dtests.slow=true -Dtests.locale=en-CA -Dtests.timezone=Greenwich -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.66s J0 | AutoCommitTest.testCommitWithin <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: Exception during query
   [junit4]    > at __randomizedtesting.SeedInfo.seed([570BC2F3989CE34B:EDD9AD8B1BB20D5E]:0)
   [junit4]    > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:878)
   [junit4]    > at org.apache.solr.update.AutoCommitTest.testCommitWithin(AutoCommitTest.java:353)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]    > xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]    > <response>
   [junit4]    > <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]    > </response>
   [junit4]    > request was:q=id:529&qt=&start=0&rows=20&version=2.2
   [junit4]    > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:871)
   [junit4]    > ... 40 more
   [junit4]   2> 1882630 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Starting testMaxTime
   [junit4]   2> 1882631 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 1882634 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5caf2bdd[collection1] realtime]
   [junit4]   2> 1882634 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 3
   [junit4]   2> 1882635 ERROR (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.CorePropertiesLocator Couldn't load core descriptor from /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/core.properties:java.nio.file.NoSuchFileException: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/core.properties
   [junit4]   2> 1882635 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/lib, /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1882669 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 1882689 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1882843 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1882850 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1
   [junit4]   2> 1882861 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41838b98
   [junit4]   2> 1882861 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.update.AutoCommitTest_570BC2F3989CE34B-001/init-core-data-001/]
   [junit4]   2> 1882920 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1882920 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1882921 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5d7d8f8[collection1] main]
   [junit4]   2> 1882922 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1882923 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1882926 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1882926 INFO  (searcherExecutor-7851-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1882927 INFO  (searcherExecutor-7851-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 1882927 INFO  (searcherExecutor-7851-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1882927 INFO  (searcherExecutor-7851-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1882927 INFO  (searcherExecutor-7851-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1882927 INFO  (searcherExecutor-7851-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 1882927 INFO  (searcherExecutor-7851-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 1882928 INFO  (searcherExecutor-7851-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 1882928 INFO  (searcherExecutor-7851-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 1882928 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=24, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1882928 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1882929 INFO  (searcherExecutor-7851-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5d7d8f8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1882930 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@359fa53d[collection1] main]
   [junit4]   2> 1882930 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@178626aa
   [junit4]   2> 1882930 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=394667690
   [junit4]   2> 1882931 INFO  (searcherExecutor-7851-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@359fa53d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1882938 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[529 (1573259221853536256)]} 0 1
   [junit4]   2> 1882939 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1884439 INFO  (commitScheduler-7854-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
   [junit4]   2> 1884440 INFO  (commitScheduler-7854-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3eba419e[collection1] main]
   [junit4]   2> 1884441 INFO  (commitScheduler-7854-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1884441 INFO  (searcherExecutor-7851-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3eba419e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_3(7.1.0):c1)))}
   [junit4]   2> 1884446 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[530 (1573259223434788864)]} 0 1509
   [junit4]   2> 1884447 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
   [junit4]   2> 1884447 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:530&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1884447 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{delete=[529 (-1573259223435837440)]} 0 0
   [junit4]   2> 1884448 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
   [junit4]   2> 1885947 INFO  (commitScheduler-7854-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
   [junit4]   2> 1885949 INFO  (commitScheduler-7854-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@84539df[collection1] main]
   [junit4]   2> 1885949 INFO  (commitScheduler-7854-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1885950 INFO  (searcherExecutor-7851-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@84539df[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_4(7.1.0):c1)))}
   [junit4]   2> 1885955 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[550 (1573259225016041472)]} 0 3018
   [junit4]   2> 1885955 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1885955 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1573259225017090048)]} 0 3018
   [junit4]   2> 1885955 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1573259225017090049)]} 0 3018
   [junit4]   2> 1885955 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1573259225017090050)]} 0 3019
   [junit4]   2> 1885956 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1573259225018138624)]} 0 3019
   [junit4]   2> 1885956 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1573259225018138625)]} 0 3019
   [junit4]   2> 1885956 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1573259225018138626)]} 0 3019
   [junit4]   2> 1885956 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1573259225018138627)]} 0 3019
   [junit4]   2> 1885956 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1573259225018138628)]} 0 3019
   [junit4]   2> 1885956 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1573259225018138629)]} 0 3019
   [junit4]   2> 1885956 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1573259225018138630)]} 0 3019
   [junit4]   2> 1885956 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:500&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1887455 INFO  (commitScheduler-7854-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
   [junit4]   2> 1887457 INFO  (commitScheduler-7854-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@39b8b3f[collection1] main]
   [junit4]   2> 1887457 INFO  (commitScheduler-7854-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1887458 INFO  (searcherExecutor-7851-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@39b8b3f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_4(7.1.0):c1) Uninverting(_5(7.1.0):c11/9:delGen=1)))}
   [junit4]   2> 1887463 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[531 (1573259226598342656)]} 0 4527
   [junit4]   2> 1887464 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:500&qt=&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
   [junit4]   2> 1887464 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[570BC2F3989CE34B]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testMaxTime
   [junit4]   2> 1887464 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 1887464 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    x:collection1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=516071134
   [junit4]   2> 1887464 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1887465 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1887466 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1887466 INFO  (SUITE-AutoCommitTest-seed#[570BC2F3989CE34B]-worker) [    x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1887466 INFO  (coreCloseExecutor-7855-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@17dd7464
   [junit4]   2> 1887466 INFO  (coreCloseExecutor-7855-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter close.
   [junit4]   2> 1887466 INFO  (coreCloseExecutor-7855-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6b2ba9b2 commitCommandVersion:0
   [junit4]   2> 1887468 INFO  (coreCloseExecutor-7855-thread-1-processing-x:collection1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=400389220
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.update.AutoCommitTest_570BC2F3989CE34B-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {field_t=PostingsFormat(name=Memory), multiDefault=PostingsFormat(name=Direct), subject=PostingsFormat(name=Direct), id=PostingsFormat(name=LuceneFixedGap), text=PostingsFormat(name=Memory)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Memory), _version_=DocValuesFormat(name=Asserting), intDefault=DocValuesFormat(name=Asserting), id_i1=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Asserting), intDvoDefault=DocValuesFormat(name=Lucene70), range_facet_l=DocValuesFormat(name=Asserting), timestamp=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=127, maxMBSortInHeap=6.48323863907337, sim=RandomSimilarity(queryNorm=true): {}, locale=en-CA, timezone=Greenwich
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=3,threads=1,free=232755720,total=530579456
   [junit4]   2> NOTE: All tests run in this JVM: [BigEndianAscendingWordDeserializerTest, MultiTermTest, PreAnalyzedUpdateProcessorTest, TestSha256AuthenticationProvider, TestUtils, TemplateUpdateProcessorTest, CursorPagingTest, DirectSolrConnectionTest, OutOfBoxZkACLAndCredentialsProvidersTest, RequestLoggingTest, TestPrepRecovery, OverseerRolesTest, CollectionsAPISolrJTest, RuleEngineTest, TestSolrConfigHandlerCloud, TestConfigSetProperties, RequestHandlersTest, TestCloudSchemaless, SharedFSAutoReplicaFailoverUtilsTest, TestStressCloudBlindAtomicUpdates, TestUninvertingReader, AddSchemaFieldsUpdateProcessorFactoryTest, TestDynamicFieldResource, HLLUtilTest, TestInfoStreamLogging, TestCSVResponseWriter, TestConfigOverlay, LeaderFailureAfterFreshStartTest, TestFieldTypeResource, HttpSolrCallGetCoreTest, CoreSorterTest, SuggesterWFSTTest, DirectUpdateHandlerOptimizeTest, TestJavabinTupleStreamParser, AnalysisAfterCoreReloadTest, TestNumericTerms32, VMParamsZkACLAndCredentialsProvidersTest, TestPayloadCheckQParserPlugin, TestCustomSort, TestHighlightDedupGrouping, ReplaceNodeTest, RegexBoostProcessorTest, SimpleFacetsTest, CdcrReplicationHandlerTest, TestSearcherReuse, NoCacheHeaderTest, BitVectorTest, TestNoOpRegenerator, TestFoldingMultitermQuery, TestLeaderElectionZkExpiry, TestSolrCoreProperties, TestLFUCache, TestStreamBody, FullHLLTest, AnalysisErrorHandlingTest, LeaderElectionTest, TestPKIAuthenticationPlugin, UtilsToolTest, SolrShardReporterTest, ResourceLoaderTest, TestApiFramework, TestReqParamsAPI, TestSchemalessBufferedUpdates, ChaosMonkeyNothingIsSafeTest, ShardSplitTest, TestPullReplicaErrorHandling, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestSurroundQueryParser, TestUniqueKeyFieldResource, TestLocalFSCloudBackupRestore, SolrTestCaseJ4Test, SolrCoreTest, ConnectionReuseTest, TestClusterStateMutator, PrimUtilsTest, TestSolrCloudSnapshots, TestTrackingShardHandlerFactory, QueryParsingTest, ClusterStateTest, InfoHandlerTest, TestFuzzyAnalyzedSuggestions, TestFunctionQuery, TestManagedSynonymFilterFactory, TestWriterPerf, UpdateLogTest, TestFieldCacheWithThreads, SolrMetricsIntegrationTest, TestXIncludeConfig, IndexBasedSpellCheckerTest, CloudMLTQParserTest, TestTolerantSearch, DefaultValueUpdateProcessorTest, TestSegmentSorting, TestXmlQParserPlugin, TestMacros, TestEmbeddedSolrServerSchemaAPI, TestPhraseSuggestions, ClusterStateUpdateTest, TestInPlaceUpdatesDistrib, HdfsRecoveryZkTest, TestMiniSolrCloudCluster, OverseerTest, DistributedFacetExistsSmallTest, DistributedFacetPivotLongTailTest, BlobRepositoryCloudTest, ShardRoutingTest, TestOrdValues, TestDeleteCollectionOnDownNodes, PropertiesRequestHandlerTest, GraphQueryTest, TestDFRSimilarityFactory, ZkCLITest, TestManagedSchemaAPI, TestSolrCloudWithKerberosAlt, OverseerTaskQueueTest, RecoveryAfterSoftCommitTest, TestSystemCollAutoCreate, InfixSuggestersTest, HighlighterMaxOffsetTest, TestSmileRequest, SolrMetricManagerTest, CoreAdminRequestStatusTest, ResponseLogComponentTest, TestStressReorder, DistributedSpellCheckComponentTest, IndexSchemaTest, DeleteNodeTest, TestSweetSpotSimilarityFactory, PKIAuthenticationIntegrationTest, TestAnalyzeInfixSuggestions, TestFileDictionaryLookup, TestSubQueryTransformer, DistributedFacetPivotSmallTest, TestExtendedDismaxParser, TestFieldResource, SolrIndexMetricsTest, TestClassicSimilarityFactory, CollectionTooManyReplicasTest, BlockJoinFacetRandomTest, CleanupOldIndexTest, CdcrVersionReplicationTest, CSVRequestHandlerTest, SoftAutoCommitTest, TestImpersonationWithHadoopAuth, TestRecoveryHdfs, ScriptEngineTest, DataDrivenBlockJoinTest, HdfsNNFailoverTest, CircularListTest, ReturnFieldsTest, DistanceUnitsTest, TestLegacyNumericUtils, HdfsTlogReplayBufferedWhileIndexingTest, SolrCLIZkUtilsTest, TestFaceting, TestHashPartitioner, TestCorePropertiesReload, SegmentsInfoRequestHandlerTest, TestHdfsCloudBackupRestore, TestSolrDeletionPolicy2, DirectUpdateHandlerTest, MissingSegmentRecoveryTest, OutputWriterTest, TestFieldCacheSortRandom, TestPointFields, SpatialRPTFieldTypeTest, TestSizeLimitedDistributedMap, TestExactStatsCache, AnalyticsQueryTest, TestBlobHandler, TestConfigsApi, TestGeoJSONResponseWriter, MinimalSchemaTest, AliasIntegrationTest, ZkStateWriterTest, TestLegacyNumericRangeQueryBuilder, TestCoreContainer, BadCopyFieldTest, TestReplicationHandler, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest, ZkSolrClientTest, TestRandomDVFaceting, AutoCommitTest]
   [junit4] Completed [373/728 (1!)] on J0 in 6.31s, 3 tests, 1 error <<< FAILURES!

[...truncated 44446 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 51 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/51/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.core.TestDynamicLoading.testDynamicLoading

Error Message:


Stack Trace:
java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([398A076889BAA3A1:E1C72A3F7E670601]:0)
        at org.junit.Assert.fail(Assert.java:92)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNotNull(Assert.java:526)
        at org.junit.Assert.assertNotNull(Assert.java:537)
        at org.apache.solr.core.TestDynamicLoading.testDynamicLoading(TestDynamicLoading.java:116)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12629 lines...]
   [junit4] Suite: org.apache.solr.core.TestDynamicLoading
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/init-core-data-001
   [junit4]   2> 2713358 WARN  (SUITE-TestDynamicLoading-seed#[398A076889BAA3A1]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
   [junit4]   2> 2713358 INFO  (SUITE-TestDynamicLoading-seed#[398A076889BAA3A1]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 2713359 INFO  (SUITE-TestDynamicLoading-seed#[398A076889BAA3A1]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2713359 INFO  (SUITE-TestDynamicLoading-seed#[398A076889BAA3A1]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /pahm/
   [junit4]   2> 2713362 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2713362 INFO  (Thread-7050) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2713362 INFO  (Thread-7050) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2713363 ERROR (Thread-7050) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2713462 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.ZkTestServer start zk server on port:34214
   [junit4]   2> 2713481 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2713484 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2713486 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2713488 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2713489 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2713492 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2713494 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2713496 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2713498 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2713501 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2713503 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2713506 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 2713729 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2713731 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5474d9d3{/pahm,null,AVAILABLE}
   [junit4]   2> 2713732 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7721027c{HTTP/1.1,[http/1.1]}{127.0.0.1:55497}
   [junit4]   2> 2713732 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.Server Started @2718065ms
   [junit4]   2> 2713733 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/tempDir-001/control/data, replicaType=NRT, hostContext=/pahm, hostPort=55497, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/control-001/cores}
   [junit4]   2> 2713733 ERROR (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2713733 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.1.0
   [junit4]   2> 2713733 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2713733 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 2713733 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-18T18:40:12.275Z
   [junit4]   2> 2713738 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2713738 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/control-001/solr.xml
   [junit4]   2> 2713752 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 2713756 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34214/solr
   [junit4]   2> 2713945 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:55497_pahm    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2713946 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:55497_pahm    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55497_pahm
   [junit4]   2> 2713949 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:55497_pahm    ] o.a.s.c.Overseer Overseer (id=98330424895406084-127.0.0.1:55497_pahm-n_0000000000) starting
   [junit4]   2> 2713974 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:55497_pahm    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55497_pahm
   [junit4]   2> 2713979 INFO  (OverseerStateUpdate-98330424895406084-127.0.0.1:55497_pahm-n_0000000000) [n:127.0.0.1:55497_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2714180 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:55497_pahm    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2714185 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:55497_pahm    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2714185 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:55497_pahm    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2714187 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:55497_pahm    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/control-001/cores
   [junit4]   2> 2714210 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2714211 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34214/solr ready
   [junit4]   2> 2714213 INFO  (qtp12077714-28361) [n:127.0.0.1:55497_pahm    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:55497_pahm&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2714218 INFO  (OverseerThreadFactory-9090-thread-1-processing-n:127.0.0.1:55497_pahm) [n:127.0.0.1:55497_pahm    ] o.a.s.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 2714329 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2714329 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 2714442 INFO  (zkCallback-4218-thread-1-processing-n:127.0.0.1:55497_pahm) [n:127.0.0.1:55497_pahm    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2715350 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 2715358 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 2715450 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2715467 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 2715468 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2715468 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2715468 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 2715469 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=46, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 2715471 WARN  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2715510 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2715510 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2715511 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2715511 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2715511 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=77.7255859375, floorSegmentMB=0.228515625, forceMergeDeletesPctAllowed=8.423835147285535, segmentsPerTier=36.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2715512 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4e7ba41a[control_collection_shard1_replica_n1] main]
   [junit4]   2> 2715513 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2715513 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2715514 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2715514 INFO  (searcherExecutor-9093-thread-1-processing-n:127.0.0.1:55497_pahm x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@4e7ba41a[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2715515 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1573286800583032832
   [junit4]   2> 2715521 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2715521 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2715522 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55497/pahm/control_collection_shard1_replica_n1/
   [junit4]   2> 2715522 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2715522 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:55497/pahm/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 2715522 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2715526 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55497/pahm/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 2715630 INFO  (zkCallback-4218-thread-1-processing-n:127.0.0.1:55497_pahm) [n:127.0.0.1:55497_pahm    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2715690 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2715693 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1364
   [junit4]   2> 2715697 INFO  (qtp12077714-28361) [n:127.0.0.1:55497_pahm    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 2715795 INFO  (zkCallback-4218-thread-1-processing-n:127.0.0.1:55497_pahm) [n:127.0.0.1:55497_pahm    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2716222 INFO  (OverseerCollectionConfigSetProcessor-98330424895406084-127.0.0.1:55497_pahm-n_0000000000) [n:127.0.0.1:55497_pahm    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2716698 INFO  (qtp12077714-28361) [n:127.0.0.1:55497_pahm    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:55497_pahm&wt=javabin&version=2} status=0 QTime=2485
   [junit4]   2> 2716708 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2716711 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34214/solr ready
   [junit4]   2> 2716712 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2716713 INFO  (qtp12077714-28360) [n:127.0.0.1:55497_pahm    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2716717 INFO  (OverseerThreadFactory-9090-thread-2-processing-n:127.0.0.1:55497_pahm) [n:127.0.0.1:55497_pahm    ] o.a.s.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 2716718 WARN  (OverseerThreadFactory-9090-thread-2-processing-n:127.0.0.1:55497_pahm) [n:127.0.0.1:55497_pahm    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 2716926 INFO  (qtp12077714-28360) [n:127.0.0.1:55497_pahm    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 2716926 INFO  (qtp12077714-28360) [n:127.0.0.1:55497_pahm    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=212
   [junit4]   2> 2717197 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-1-001 of type NRT
   [junit4]   2> 2717198 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2717199 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@63b9b445{/pahm,null,AVAILABLE}
   [junit4]   2> 2717199 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@12c35b7e{HTTP/1.1,[http/1.1]}{127.0.0.1:39769}
   [junit4]   2> 2717199 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.Server Started @2721533ms
   [junit4]   2> 2717199 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/tempDir-001/jetty1, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/pahm, hostPort=39769, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-1-001/cores}
   [junit4]   2> 2717200 ERROR (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2717200 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.1.0
   [junit4]   2> 2717200 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2717200 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 2717200 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-18T18:40:15.742Z
   [junit4]   2> 2717220 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2717220 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-1-001/solr.xml
   [junit4]   2> 2717225 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 2717228 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34214/solr
   [junit4]   2> 2717265 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:39769_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2717273 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:39769_pahm    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2717277 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:39769_pahm    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39769_pahm
   [junit4]   2> 2717280 INFO  (zkCallback-4218-thread-1-processing-n:127.0.0.1:55497_pahm) [n:127.0.0.1:55497_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2717280 INFO  (zkCallback-4225-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2717281 INFO  (zkCallback-4230-thread-1-processing-n:127.0.0.1:39769_pahm) [n:127.0.0.1:39769_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2717395 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:39769_pahm    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2717400 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:39769_pahm    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2717401 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:39769_pahm    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2717402 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:39769_pahm    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-1-001/cores
   [junit4]   2> 2717438 INFO  (qtp12077714-28359) [n:127.0.0.1:55497_pahm    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:39769_pahm&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2717444 INFO  (OverseerCollectionConfigSetProcessor-98330424895406084-127.0.0.1:55497_pahm-n_0000000000) [n:127.0.0.1:55497_pahm    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2717444 INFO  (OverseerThreadFactory-9090-thread-3-processing-n:127.0.0.1:55497_pahm) [n:127.0.0.1:55497_pahm    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:39769_pahm for creating new replica
   [junit4]   2> 2717446 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n0&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2717447 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 2718467 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 2718486 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.s.IndexSchema [collection1_shard2_replica_n0] Schema name=test
   [junit4]   2> 2718602 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2718654 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n0' using configuration from collection collection1, trusted=true
   [junit4]   2> 2718655 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n0' (registry 'solr.core.collection1.shard2.replica_n0') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2718655 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2718655 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.SolrCore [[collection1_shard2_replica_n0] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-1-001/cores/collection1_shard2_replica_n0], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-1-001/cores/collection1_shard2_replica_n0/data/]
   [junit4]   2> 2718657 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=46, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 2718660 WARN  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2718724 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2718724 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2718726 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2718726 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2718727 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=77.7255859375, floorSegmentMB=0.228515625, forceMergeDeletesPctAllowed=8.423835147285535, segmentsPerTier=36.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2718727 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.s.SolrIndexSearcher Opening [Searcher@5e25a3d[collection1_shard2_replica_n0] main]
   [junit4]   2> 2718729 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2718729 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2718730 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2718730 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1573286803954204672
   [junit4]   2> 2718731 INFO  (searcherExecutor-9104-thread-1-processing-n:127.0.0.1:39769_pahm x:collection1_shard2_replica_n0 s:shard2 c:collection1) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.SolrCore [collection1_shard2_replica_n0] Registered new searcher Searcher@5e25a3d[collection1_shard2_replica_n0] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2718742 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2718742 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2718743 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39769/pahm/collection1_shard2_replica_n0/
   [junit4]   2> 2718743 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2718743 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.SyncStrategy http://127.0.0.1:39769/pahm/collection1_shard2_replica_n0/ has no replicas
   [junit4]   2> 2718743 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2718750 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39769/pahm/collection1_shard2_replica_n0/ shard2
   [junit4]   2> 2718902 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2718904 INFO  (qtp706628445-28405) [n:127.0.0.1:39769_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n0&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1458
   [junit4]   2> 2718908 INFO  (qtp12077714-28359) [n:127.0.0.1:55497_pahm    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:39769_pahm&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1470
   [junit4]   2> 2719041 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-2-001 of type NRT
   [junit4]   2> 2719041 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2719042 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@67da0bce{/pahm,null,AVAILABLE}
   [junit4]   2> 2719043 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@50df255f{HTTP/1.1,[http/1.1]}{127.0.0.1:50647}
   [junit4]   2> 2719043 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.Server Started @2723377ms
   [junit4]   2> 2719043 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/pahm, hostPort=50647, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-2-001/cores}
   [junit4]   2> 2719043 ERROR (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2719043 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.1.0
   [junit4]   2> 2719043 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2719043 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 2719043 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-18T18:40:17.585Z
   [junit4]   2> 2719055 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2719055 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-2-001/solr.xml
   [junit4]   2> 2719075 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 2719093 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34214/solr
   [junit4]   2> 2719184 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:50647_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2719186 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:50647_pahm    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2719188 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:50647_pahm    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50647_pahm
   [junit4]   2> 2719190 INFO  (zkCallback-4218-thread-1-processing-n:127.0.0.1:55497_pahm) [n:127.0.0.1:55497_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2719190 INFO  (zkCallback-4225-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2719192 INFO  (zkCallback-4230-thread-1-processing-n:127.0.0.1:39769_pahm) [n:127.0.0.1:39769_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2719237 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:50647_pahm    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2719240 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:50647_pahm    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2719240 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:50647_pahm    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2719241 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:50647_pahm    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-2-001/cores
   [junit4]   2> 2719305 INFO  (zkCallback-4236-thread-1-processing-n:127.0.0.1:50647_pahm) [n:127.0.0.1:50647_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2719328 INFO  (qtp12077714-28358) [n:127.0.0.1:55497_pahm    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:50647_pahm&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2719360 INFO  (OverseerThreadFactory-9090-thread-4-processing-n:127.0.0.1:55497_pahm) [n:127.0.0.1:55497_pahm    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:50647_pahm for creating new replica
   [junit4]   2> 2719361 INFO  (OverseerCollectionConfigSetProcessor-98330424895406084-127.0.0.1:55497_pahm-n_0000000000) [n:127.0.0.1:55497_pahm    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2719373 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n0&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2719374 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 2720395 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 2720408 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.s.IndexSchema [collection1_shard1_replica_n0] Schema name=test
   [junit4]   2> 2720484 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2720514 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n0' using configuration from collection collection1, trusted=true
   [junit4]   2> 2720514 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n0' (registry 'solr.core.collection1.shard1.replica_n0') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2720514 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2720514 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.SolrCore [[collection1_shard1_replica_n0] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-2-001/cores/collection1_shard1_replica_n0], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-2-001/cores/collection1_shard1_replica_n0/data/]
   [junit4]   2> 2720516 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=46, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 2720520 WARN  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2720589 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2720589 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2720591 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2720591 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2720592 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=77.7255859375, floorSegmentMB=0.228515625, forceMergeDeletesPctAllowed=8.423835147285535, segmentsPerTier=36.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2720592 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.s.SolrIndexSearcher Opening [Searcher@3b4dde91[collection1_shard1_replica_n0] main]
   [junit4]   2> 2720594 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2720594 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2720595 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2720596 INFO  (searcherExecutor-9115-thread-1-processing-n:127.0.0.1:50647_pahm x:collection1_shard1_replica_n0 s:shard1 c:collection1) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.SolrCore [collection1_shard1_replica_n0] Registered new searcher Searcher@3b4dde91[collection1_shard1_replica_n0] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2720596 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1573286805910847488
   [junit4]   2> 2720607 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2720607 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2720607 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:50647/pahm/collection1_shard1_replica_n0/
   [junit4]   2> 2720607 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2720607 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.SyncStrategy http://127.0.0.1:50647/pahm/collection1_shard1_replica_n0/ has no replicas
   [junit4]   2> 2720607 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2720613 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:50647/pahm/collection1_shard1_replica_n0/ shard1
   [junit4]   2> 2720766 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2720770 INFO  (qtp1824041799-28433) [n:127.0.0.1:50647_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n0&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1398
   [junit4]   2> 2720776 INFO  (qtp12077714-28358) [n:127.0.0.1:55497_pahm    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:50647_pahm&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1449
   [junit4]   2> 2720913 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-3-001 of type NRT
   [junit4]   2> 2720914 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2720915 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@53f4009d{/pahm,null,AVAILABLE}
   [junit4]   2> 2720916 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@792641d7{HTTP/1.1,[http/1.1]}{127.0.0.1:62499}
   [junit4]   2> 2720916 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.Server Started @2725251ms
   [junit4]   2> 2720916 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/tempDir-001/jetty3, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/pahm, hostPort=62499, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-3-001/cores}
   [junit4]   2> 2720917 ERROR (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2720917 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.1.0
   [junit4]   2> 2720917 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2720917 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 2720917 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-18T18:40:19.459Z
   [junit4]   2> 2720920 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2720920 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-3-001/solr.xml
   [junit4]   2> 2720923 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 2720925 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34214/solr
   [junit4]   2> 2720940 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:62499_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 2720944 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:62499_pahm    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2720949 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:62499_pahm    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62499_pahm
   [junit4]   2> 2720953 INFO  (zkCallback-4218-thread-1-processing-n:127.0.0.1:55497_pahm) [n:127.0.0.1:55497_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2720953 INFO  (zkCallback-4225-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2720954 INFO  (zkCallback-4230-thread-1-processing-n:127.0.0.1:39769_pahm) [n:127.0.0.1:39769_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2720954 INFO  (zkCallback-4236-thread-1-processing-n:127.0.0.1:50647_pahm) [n:127.0.0.1:50647_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2720965 INFO  (zkCallback-4242-thread-1-processing-n:127.0.0.1:62499_pahm) [n:127.0.0.1:62499_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2721063 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:62499_pahm    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2721068 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:62499_pahm    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2721068 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:62499_pahm    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2721069 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:62499_pahm    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-3-001/cores
   [junit4]   2> 2721099 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:62499_pahm&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2721102 INFO  (OverseerThreadFactory-9090-thread-5-processing-n:127.0.0.1:55497_pahm) [n:127.0.0.1:55497_pahm    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:62499_pahm for creating new replica
   [junit4]   2> 2721103 INFO  (OverseerCollectionConfigSetProcessor-98330424895406084-127.0.0.1:55497_pahm-n_0000000000) [n:127.0.0.1:55497_pahm    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2721105 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n1&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2721105 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 2722123 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 2722135 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.s.IndexSchema [collection1_shard2_replica_n1] Schema name=test
   [junit4]   2> 2722223 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2722264 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n1' using configuration from collection collection1, trusted=true
   [junit4]   2> 2722264 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n1' (registry 'solr.core.collection1.shard2.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2722264 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2722264 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.c.SolrCore [[collection1_shard2_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-3-001/cores/collection1_shard2_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-3-001/cores/collection1_shard2_replica_n1/data/]
   [junit4]   2> 2722266 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=46, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 2722273 WARN  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2722343 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2722343 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2722343 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2722343 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2722343 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=77.7255859375, floorSegmentMB=0.228515625, forceMergeDeletesPctAllowed=8.423835147285535, segmentsPerTier=36.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2722343 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@36b4fb8[collection1_shard2_replica_n1] main]
   [junit4]   2> 2722348 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2722349 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2722349 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2722350 INFO  (searcherExecutor-9126-thread-1-processing-n:127.0.0.1:62499_pahm x:collection1_shard2_replica_n1 s:shard2 c:collection1) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.c.SolrCore [collection1_shard2_replica_n1] Registered new searcher Searcher@36b4fb8[collection1_shard2_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2722350 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1573286807750049792
   [junit4]   2> 2722355 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.c.ZkController Core needs to recover:collection1_shard2_replica_n1
   [junit4]   2> 2722356 INFO  (updateExecutor-4239-thread-1-processing-n:127.0.0.1:62499_pahm x:collection1_shard2_replica_n1 s:shard2 c:collection1) [n:127.0.0.1:62499_pahm c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 2722357 INFO  (qtp558508592-28464) [n:127.0.0.1:62499_pahm c:collection1 s:shard2  x:collection1_shard2_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n1&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1252
   [junit4]   2> 2722357 INFO  (recoveryExecutor-4240-thread-1-processing-n:127.0.0.1:62499_pahm x:collection1_shard2_replica_n1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:62499_pahm c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 2722358 INFO  (recoveryExecutor-4240-thread-1-processing-n:127.0.0.1:62499_pahm x:collection1_shard2_replica_n1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:62499_pahm c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 2722358 INFO  (recoveryExecutor-4240-thread-1-processing-n:127.0.0.1:62499_pahm x:collection1_shard2_replica_n1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:62499_pahm c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard2_replica_n1]
   [junit4]   2> 2722358 INFO  (recoveryExecutor-4240-thread-1-processing-n:127.0.0.1:62499_pahm x:collection1_shard2_replica_n1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:62499_pahm c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 2722358 INFO  (recoveryExecutor-4240-thread-1-processing-n:127.0.0.1:62499_pahm x:collection1_shard2_replica_n1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:62499_pahm c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard2_replica_n1] as recovering, leader is [http://127.0.0.1:39769/pahm/collection1_shard2_replica_n0/] and I am [http://127.0.0.1:62499/pahm/collection1_shard2_replica_n1/]
   [junit4]   2> 2722360 INFO  (qtp12077714-28363) [n:127.0.0.1:55497_pahm    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:62499_pahm&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1261
   [junit4]   2> 2722362 INFO  (recoveryExecutor-4240-thread-1-processing-n:127.0.0.1:62499_pahm x:collection1_shard2_replica_n1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:62499_pahm c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:39769/pahm]; [WaitForState: action=PREPRECOVERY&core=collection1_shard2_replica_n0&nodeName=127.0.0.1:62499_pahm&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 2722363 INFO  (qtp706628445-28403) [n:127.0.0.1:39769_pahm    ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
   [junit4]   2> 2722364 INFO  (qtp706628445-28403) [n:127.0.0.1:39769_pahm    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1_shard2_replica_n0, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:62499_pahm, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1_shard2_replica_n1","base_url":"http://127.0.0.1:62499/pahm","node_name":"127.0.0.1:62499_pahm","state":"down","type":"NRT"}
   [junit4]   2> 2722564 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-4-001 of type NRT
   [junit4]   2> 2722564 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2722567 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5f66ef51{/pahm,null,AVAILABLE}
   [junit4]   2> 2722572 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7be613c{HTTP/1.1,[http/1.1]}{127.0.0.1:64849}
   [junit4]   2> 2722573 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.Server Started @2726908ms
   [junit4]   2> 2722573 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/tempDir-001/jetty4, solrconfig=solrconfig.xml, hostContext=/pahm, hostPort=64849, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-4-001/cores}
   [junit4]   2> 2722577 ERROR (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2722577 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.1.0
   [junit4]   2> 2722577 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2722577 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 2722577 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-18T18:40:21.119Z
   [junit4]   2> 2722581 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2722581 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-4-001/solr.xml
   [junit4]   2> 2722586 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 2722589 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34214/solr
   [junit4]   2> 2722603 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:64849_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 2722608 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:64849_pahm    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2722610 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:64849_pahm    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64849_pahm
   [junit4]   2> 2722612 INFO  (zkCallback-4225-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2722612 INFO  (zkCallback-4236-thread-1-processing-n:127.0.0.1:50647_pahm) [n:127.0.0.1:50647_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2722613 INFO  (zkCallback-4242-thread-1-processing-n:127.0.0.1:62499_pahm) [n:127.0.0.1:62499_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2722613 INFO  (zkCallback-4218-thread-1-processing-n:127.0.0.1:55497_pahm) [n:127.0.0.1:55497_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2722613 INFO  (zkCallback-4230-thread-1-processing-n:127.0.0.1:39769_pahm) [n:127.0.0.1:39769_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2722613 INFO  (zkCallback-4249-thread-1-processing-n:127.0.0.1:64849_pahm) [n:127.0.0.1:64849_pahm    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 2722769 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:64849_pahm    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2722772 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:64849_pahm    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2722773 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:64849_pahm    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@737c1d0c
   [junit4]   2> 2722774 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [n:127.0.0.1:64849_pahm    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001/shard-4-001/cores
   [junit4]   2> 2722800 INFO  (qtp12077714-28361) [n:127.0.0.1:55497_pahm    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:64849_pahm&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2722803 INFO  (OverseerThreadFactory-9090-thread-5-processing-n:127.0.0.1:55497_pahm) [n:127.0.0.1:55497_pahm    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:64849_pahm for creating new replica
   [junit4]   2> 2722804 INFO  (OverseerCollectionConfigSetProcessor-98330424895406084-127.0.0.1:55497_pahm-n_0000000000) [n:127.0.0.1:55497_pahm    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2722805 INFO  (qtp1438525108-28498) [n:127.0.0.1:64849_pahm    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n1&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2722806 INFO  (qtp1438525108-28498) [n:127.0.0.1:64849_pahm    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 2723365 INFO  (qtp706628445-28403) [n:127.0.0.1:39769_pahm    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1_shard2_replica_n0, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:62499_pahm, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1_shard2_replica_n1","base_url":"http://127.0.0.1:62499/pahm","node_name":"127.0.0.1:62499_pahm","state":"recovering","type":"NRT"}
   [junit4]   2> 2723365 INFO  (qtp706628445-28403) [n:127.0.0.1:39769_pahm    ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
   [junit4]   2> 2723365 INFO  (qtp706628445-28403) [n:127.0.0.1:39769_pahm    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:62499_pahm&onlyIfLeaderActive=true&core=collection1_shard2_replica_n0&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1002
   [junit4]   2> 2723829 INFO  (qtp1438525108-28498) [n:127.0.0.1:64849_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
   [junit4]   2> 2723838 INFO  (qtp1438525108-28498) [n:127.0.0.1:64849_pahm c:collection1 s:shard1  x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema [collection1_shard1_replica_n1] Schema name=test
   [junit4]   2> 2723875 INFO  (recoveryExecutor-4240-thread-1-processing-n:127.0.0.1:62499_pahm x:collection1_shard2_replica_n1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:62499_pahm c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:39769/pahm/collection1_shard2_replica_n0/] - recoveringAfterStartup=[true]
   [junit4]   2> 2723875 INFO  (recoveryExecutor-4240-thread-1-processing-n:127.0.0.1:62499_pahm x:collection1_shard2_replica_n1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:62499_pahm c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.PeerSync PeerSync: core=collection1_shard2_replica_n1 url=http://127.0.0.1:62499/pahm START replicas=[http://127.0.0.1:39769/pahm/collection1_shard2_replica_n0/] nUpdates=100
   [junit4]   2> 2723878 INFO  (qtp706628445-28407) [n:127.0.0.1:39769_pahm c:collection1 s:shard2 r:core_node1 x:collection1_shard2_replica_n0] o.a.s.u.IndexFingerprint IndexFingerprint millis:1.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
   [junit4]   2> 2723878 INFO  (qtp706628445-28407) [n:127.0.0.1:39769_pahm c:collection1 s:shard2 r:core_node1 x:collection1_shard2_replica_n0] o.a.s.c.S.Request [collection1_shard2_replica_n0]  webapp=/pahm path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 2723879 INFO  (recoveryExecutor-4240-thread-1-processing-n:127.0.0.1:62499_pahm x:collection1_shard2_replica_n1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:62499_pahm c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
   [junit4]   2> 2723879 INFO  (recoveryExecutor-4240-thread-1-processing-n:127.0.0.1:62499_pahm x:collection1_shard2_replica_n1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:62499_pahm c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
   [junit4]   2> 2723879 INFO  (recoveryExecutor-4240-thread-1-processing-n:127.0.0.1:62499_pahm x:collection1_shard2_replica_n1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:62499_pahm c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2723879 INFO  (recoveryExecutor-4240-thread-1-processing-n:127.0.0.1:62499_pahm x:collection1_shard2_replica_n1 s:sh

[...truncated too long message...]

 c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 2732471 INFO  (zkCallback-4249-thread-2-processing-n:127.0.0.1:64849_pahm) [n:127.0.0.1:64849_pahm c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [collection1_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4fcc879d
   [junit4]   2> 2732472 INFO  (zkCallback-4249-thread-2-processing-n:127.0.0.1:64849_pahm) [n:127.0.0.1:64849_pahm c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_n1, tag=1338804125
   [junit4]   2> 2732472 INFO  (zkCallback-4249-thread-2-processing-n:127.0.0.1:64849_pahm) [n:127.0.0.1:64849_pahm c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=1338804125
   [junit4]   2> 2732473 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@7be613c{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 2732473 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5f66ef51{/pahm,null,UNAVAILABLE}
   [junit4]   2> 2732475 ERROR (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2732477 INFO  (TEST-TestDynamicLoading.testDynamicLoading-seed#[398A076889BAA3A1]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34214 34214
   [junit4]   2> 2733299 INFO  (Thread-7050) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34214 34214
   [junit4]   2> 2733300 WARN  (Thread-7050) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 15 /solr/configs/conf1
   [junit4]   2> 7 /solr/aliases.json
   [junit4]   2> 5 /solr/security.json
   [junit4]   2> 4 /solr/collections/collection1/state.json
   [junit4]   2> 2 /solr/configs/.system
   [junit4]   2> 2 /solr/configs/.system/managed-schema
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 7 /solr/clusterstate.json
   [junit4]   2> 7 /solr/clusterprops.json
   [junit4]   2> 2 /solr/collections/.system/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 7 /solr/live_nodes
   [junit4]   2> 7 /solr/collections
   [junit4]   2> 3 /solr/overseer/collection-queue-work
   [junit4]   2> 2 /solr/overseer/queue
   [junit4]   2>
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDynamicLoading -Dtests.method=testDynamicLoading -Dtests.seed=398A076889BAA3A1 -Dtests.slow=true -Dtests.locale=ga -Dtests.timezone=Chile/Continental -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 19.9s J0 | TestDynamicLoading.testDynamicLoading <<<
   [junit4]    > Throwable #1: java.lang.AssertionError
   [junit4]    > at __randomizedtesting.SeedInfo.seed([398A076889BAA3A1:E1C72A3F7E670601]:0)
   [junit4]    > at org.apache.solr.core.TestDynamicLoading.testDynamicLoading(TestDynamicLoading.java:116)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.core.TestDynamicLoading_398A076889BAA3A1-001
   [junit4]   2> Jul 18, 2017 6:40:31 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1628, maxMBSortInHeap=5.784547253681907, sim=RandomSimilarity(queryNorm=false): {}, locale=ga, timezone=Chile/Continental
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=3,threads=1,free=73955248,total=530579456
   [junit4]   2> NOTE: All tests run in this JVM: [TestSubQueryTransformerDistrib, UnloadDistributedZkTest, LeaderInitiatedRecoveryOnShardRestartTest, CollectionTooManyReplicasTest, DirectUpdateHandlerTest, TestUpdate, TestFuzzyAnalyzedSuggestions, BlockDirectoryTest, AsyncCallRequestStatusResponseTest, TestCursorMarkWithoutUniqueKey, ZkStateWriterTest, SolrCLIZkUtilsTest, QueryParsingTest, TestSolrConfigHandlerCloud, AtomicUpdatesTest, PreAnalyzedFieldManagedSchemaCloudTest, TestFileDictionaryLookup, TestSolrDeletionPolicy2, TestStressCloudBlindAtomicUpdates, CachingDirectoryFactoryTest, OverseerTaskQueueTest, DateRangeFieldTest, TestSearcherReuse, HdfsTlogReplayBufferedWhileIndexingTest, TestPointFields, TestNumericTerms32, XsltUpdateRequestHandlerTest, SolrIndexMetricsTest, TestQuerySenderListener, TestCustomDocTransformer, ForceLeaderTest, TestSchemaResource, TestClusterStateMutator, CircularListTest, TestOrdValues, ClusterStateUpdateTest, TestInPlaceUpdatesDistrib, TestLMDirichletSimilarityFactory, TestDocTermOrds, BasicDistributedZkTest, TestSort, TestLFUCache, HighlighterMaxOffsetTest, HdfsWriteToMultipleCollectionsTest, TestShardHandlerFactory, CreateCollectionCleanupTest, TestRecoveryHdfs, TestLocalFSCloudBackupRestore, AnalysisAfterCoreReloadTest, CdcrReplicationHandlerTest, TestDistribIDF, TestTolerantUpdateProcessorCloud, DistribCursorPagingTest, TestEmbeddedSolrServerSchemaAPI, AliasIntegrationTest, TestDFRSimilarityFactory, DistributedVersionInfoTest, NumericFieldsTest, TestCSVResponseWriter, TestDistributedSearch, CoreSorterTest, PreAnalyzedUpdateProcessorTest, TestIndexingPerformance, TestCloudPseudoReturnFields, TestSystemCollAutoCreate, SolrXmlInZkTest, SolrGraphiteReporterTest, ClassificationUpdateProcessorIntegrationTest, BasicAuthIntegrationTest, TestStressLiveNodes, CoreAdminRequestStatusTest, TestSha256AuthenticationProvider, TestLeaderElectionZkExpiry, TestSchemaVersionResource, TestLegacyNumericRangeQueryBuilder, TestHashPartitioner, TestCorePropertiesReload, TestOmitPositions, CoreAdminHandlerTest, MoveReplicaHDFSUlogDirTest, TestLegacyNumericUtils, AlternateDirectoryTest, TestHdfsBackupRestoreCore, TestFieldTypeResource, TestReplicaProperties, IgnoreCommitOptimizeUpdateProcessorFactoryTest, CursorPagingTest, TestFunctionQuery, BJQParserTest, TestConfigOverlay, TestConfigsApi, TestSolrConfigHandler, CollectionsAPISolrJTest, PeerSyncTest, TemplateUpdateProcessorTest, WordBreakSolrSpellCheckerTest, TestHdfsCloudBackupRestore, MultiThreadedOCPTest, OverseerTest, PathHierarchyTokenizerFactoryTest, AnalysisErrorHandlingTest, VMParamsZkACLAndCredentialsProvidersTest, TestReplicationHandler, TestCloudJSONFacetJoinDomain, ResponseLogComponentTest, TestSweetSpotSimilarityFactory, TestFieldResource, TestCustomSort, TestSchemalessBufferedUpdates, NoCacheHeaderTest, HdfsChaosMonkeySafeLeaderTest, TestFoldingMultitermQuery, BlobRepositoryCloudTest, SoftAutoCommitTest, TestTrieFacet, MetricsConfigTest, RequestLoggingTest, HighlighterTest, TestStressRecovery, NotRequiredUniqueKeyTest, RankQueryTest, TestCoreContainer, DistanceUnitsTest, ZkCLITest, TestDistributedStatsComponentCardinality, ExitableDirectoryReaderTest, TestRandomRequestDistribution, CloudMLTQParserTest, TestJmxIntegration, TestUninvertingReader, ConjunctionSolrSpellCheckerTest, RecoveryZkTest, RequestHandlersTest, TestPseudoReturnFields, EnumFieldTest, TestRandomDVFaceting, TestRawResponseWriter, DistributedFacetPivotSmallTest, HardAutoCommitTest, UniqFieldsUpdateProcessorFactoryTest, HighlighterConfigTest, BigEndianAscendingWordDeserializerTest, TestFieldCacheSortRandom, FieldAnalysisRequestHandlerTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestJavabinTupleStreamParser, DistributedFacetExistsSmallTest, MetricUtilsTest, TestPhraseSuggestions, MultiTermTest, LeaderFailureAfterFreshStartTest, TestStressVersions, TestClassicSimilarityFactory, SpatialRPTFieldTypeTest, TestPKIAuthenticationPlugin, TestAnalyzeInfixSuggestions, SolrTestCaseJ4Test, TestQueryWrapperFilter, TestUtils, DistributedDebugComponentTest, V2StandaloneTest, SolrMetricsIntegrationTest, TestSolrCoreParser, SolrShardReporterTest, TestNonDefinedSimilarityFactory, TestQuerySenderNoQuery, RecoveryAfterSoftCommitTest, TestScoreJoinQPScore, TestSolrCloudSnapshots, PrimUtilsTest, RAMDirectoryFactoryTest, UpdateLogTest, TestLazyCores, OverseerCollectionConfigSetProcessorTest, ReturnFieldsTest, SharedFSAutoReplicaFailoverTest, SharedFSAutoReplicaFailoverUtilsTest, SimpleCollectionCreateDeleteTest, SolrCloudExampleTest, TestCloudDeleteByQuery, TestCloudPivotFacet, TestCloudRecovery, TestClusterProperties, TestConfigSetsAPI, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestDownShardTolerantSearch, TestExclusionRuleCollectionAccess, TestLeaderElectionWithEmptyReplica, TestMiniSolrCloudCluster, TestPrepRecovery, TestRequestStatusCollectionAPI, TestSizeLimitedDistributedMap, TestTlogReplica, HdfsRecoveryZkTest, ZkStateReaderTest, RuleEngineTest, DirectoryFactoryTest, QueryResultKeyTest, TestBackupRepositoryFactory, TestConfigSetImmutable, TestConfigSetProperties, TestConfigSets, TestDynamicLoading]
   [junit4] Completed [568/728 (1!)] on J0 in 20.21s, 1 test, 1 failure <<< FAILURES!

[...truncated 43882 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Loading...