[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1275 - Failure

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1275 - Failure

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1275/

2 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test

Error Message:
document count mismatch.  control=10164 sum(shards)=10163 cloudClient=10163

Stack Trace:
java.lang.AssertionError: document count mismatch.  control=10164 sum(shards)=10163 cloudClient=10163
        at __randomizedtesting.SeedInfo.seed([D4517FC2A0E521C4:5C0540180E194C3C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1337)
        at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:236)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.security.BasicAuthIntegrationTest.testBasicAuth

Error Message:
expected:<200> but was:<403>

Stack Trace:
java.lang.AssertionError: expected:<200> but was:<403>
        at __randomizedtesting.SeedInfo.seed([D4517FC2A0E521C4:683F09D004B6A2BE]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.security.BasicAuthIntegrationTest.executeCommand(BasicAuthIntegrationTest.java:235)
        at org.apache.solr.security.BasicAuthIntegrationTest.testBasicAuth(BasicAuthIntegrationTest.java:147)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12384 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/init-core-data-001
   [junit4]   2> 2333308 WARN  (SUITE-ChaosMonkeyNothingIsSafeTest-seed#[D4517FC2A0E521C4]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 2333309 INFO  (SUITE-ChaosMonkeyNothingIsSafeTest-seed#[D4517FC2A0E521C4]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields
   [junit4]   2> 2333310 INFO  (SUITE-ChaosMonkeyNothingIsSafeTest-seed#[D4517FC2A0E521C4]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 2333310 INFO  (SUITE-ChaosMonkeyNothingIsSafeTest-seed#[D4517FC2A0E521C4]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /co_lw/
   [junit4]   2> 2333313 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2333313 INFO  (Thread-9056) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2333313 INFO  (Thread-9056) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2333413 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.ZkTestServer start zk server on port:33250
   [junit4]   2> 2333424 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2333426 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 2333427 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2333428 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2333430 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2333430 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2333431 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2333432 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2333433 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2333434 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2333435 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2333504 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/control-001/cores/collection1
   [junit4]   2> 2333505 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2333506 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@65e5d9b8{/co_lw,null,AVAILABLE}
   [junit4]   2> 2333507 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2cc11e35{HTTP/1.1,[http/1.1]}{127.0.0.1:52852}
   [junit4]   2> 2333508 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.Server Started @2336458ms
   [junit4]   2> 2333508 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/tempDir-001/control/data, hostContext=/co_lw, hostPort=52852, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/control-001/cores}
   [junit4]   2> 2333508 ERROR (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2333508 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 2333508 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2333508 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2333508 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-03-28T07:47:57.697Z
   [junit4]   2> 2333510 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2333510 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/control-001/solr.xml
   [junit4]   2> 2333524 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33250/solr
   [junit4]   2> 2333550 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:52852_co_lw    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2333551 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:52852_co_lw    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52852_co_lw
   [junit4]   2> 2333552 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:52852_co_lw    ] o.a.s.c.Overseer Overseer (id=97693681418436612-127.0.0.1:52852_co_lw-n_0000000000) starting
   [junit4]   2> 2333556 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:52852_co_lw    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52852_co_lw
   [junit4]   2> 2333558 INFO  (zkCallback-33215-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2333614 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:52852_co_lw    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/control-001/cores
   [junit4]   2> 2333614 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:52852_co_lw    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 2333617 INFO  (OverseerStateUpdate-97693681418436612-127.0.0.1:52852_co_lw-n_0000000000) [n:127.0.0.1:52852_co_lw    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2333646 WARN  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2333647 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2333666 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2333809 WARN  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 2333810 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 2333827 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2333827 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2333827 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/control-001/cores/collection1/data/]
   [junit4]   2> 2333827 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c6651ed
   [junit4]   2> 2333829 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=50, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=2.126953125, noCFSRatio=0.13662606967834395]
   [junit4]   2> 2333842 WARN  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2333886 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2333886 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2333887 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
   [junit4]   2> 2333887 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2333887 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=27, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.0009765625, noCFSRatio=0.0]
   [junit4]   2> 2333888 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@30f8767e[collection1] main]
   [junit4]   2> 2333889 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2333889 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2333889 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2333890 INFO  (coreLoadExecutor-11732-thread-1-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1563098903298965504
   [junit4]   2> 2333893 INFO  (searcherExecutor-11733-thread-1-processing-n:127.0.0.1:52852_co_lw x:collection1 c:control_collection) [n:127.0.0.1:52852_co_lw c:control_collection   x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@30f8767e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2333898 INFO  (coreZkRegister-11725-thread-1-processing-n:127.0.0.1:52852_co_lw x:collection1 c:control_collection) [n:127.0.0.1:52852_co_lw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2333898 INFO  (coreZkRegister-11725-thread-1-processing-n:127.0.0.1:52852_co_lw x:collection1 c:control_collection) [n:127.0.0.1:52852_co_lw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2333898 INFO  (coreZkRegister-11725-thread-1-processing-n:127.0.0.1:52852_co_lw x:collection1 c:control_collection) [n:127.0.0.1:52852_co_lw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52852/co_lw/collection1/
   [junit4]   2> 2333899 INFO  (coreZkRegister-11725-thread-1-processing-n:127.0.0.1:52852_co_lw x:collection1 c:control_collection) [n:127.0.0.1:52852_co_lw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2333899 INFO  (coreZkRegister-11725-thread-1-processing-n:127.0.0.1:52852_co_lw x:collection1 c:control_collection) [n:127.0.0.1:52852_co_lw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:52852/co_lw/collection1/ has no replicas
   [junit4]   2> 2333899 INFO  (coreZkRegister-11725-thread-1-processing-n:127.0.0.1:52852_co_lw x:collection1 c:control_collection) [n:127.0.0.1:52852_co_lw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2333901 INFO  (coreZkRegister-11725-thread-1-processing-n:127.0.0.1:52852_co_lw x:collection1 c:control_collection) [n:127.0.0.1:52852_co_lw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52852/co_lw/collection1/ shard1
   [junit4]   2> 2333990 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2333991 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33250/solr ready
   [junit4]   2> 2333991 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2333991 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 2334002 INFO  (coreZkRegister-11725-thread-1-processing-n:127.0.0.1:52852_co_lw x:collection1 c:control_collection) [n:127.0.0.1:52852_co_lw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2334065 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-1-001/cores/collection1
   [junit4]   2> 2334065 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-1-001
   [junit4]   2> 2334066 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2334067 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1c1742c2{/co_lw,null,AVAILABLE}
   [junit4]   2> 2334068 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@34e5b460{HTTP/1.1,[http/1.1]}{127.0.0.1:58206}
   [junit4]   2> 2334068 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.Server Started @2337017ms
   [junit4]   2> 2334068 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/co_lw, hostPort=58206, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-1-001/cores}
   [junit4]   2> 2334068 ERROR (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2334068 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 2334068 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2334068 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2334068 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-03-28T07:47:58.257Z
   [junit4]   2> 2334070 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2334070 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-1-001/solr.xml
   [junit4]   2> 2334076 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33250/solr
   [junit4]   2> 2334087 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:58206_co_lw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2334089 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:58206_co_lw    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2334090 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:58206_co_lw    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58206_co_lw
   [junit4]   2> 2334091 INFO  (zkCallback-33215-thread-2-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2334091 INFO  (zkCallback-33219-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2334092 INFO  (zkCallback-33225-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2334202 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:58206_co_lw    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-1-001/cores
   [junit4]   2> 2334203 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:58206_co_lw    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 2334208 INFO  (OverseerStateUpdate-97693681418436612-127.0.0.1:52852_co_lw-n_0000000000) [n:127.0.0.1:52852_co_lw    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2334309 INFO  (zkCallback-33225-thread-2-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2335218 WARN  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2335219 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2335237 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2335315 WARN  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 2335317 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 2335327 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 2335328 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2335328 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-1-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-1-001/cores/collection1/data/]
   [junit4]   2> 2335328 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c6651ed
   [junit4]   2> 2335330 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=50, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=2.126953125, noCFSRatio=0.13662606967834395]
   [junit4]   2> 2335339 WARN  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2335366 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2335366 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2335367 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
   [junit4]   2> 2335367 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2335368 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=27, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.0009765625, noCFSRatio=0.0]
   [junit4]   2> 2335368 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@43e9b1e3[collection1] main]
   [junit4]   2> 2335369 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2335370 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2335370 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2335371 INFO  (searcherExecutor-11744-thread-1-processing-n:127.0.0.1:58206_co_lw x:collection1 c:collection1) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@43e9b1e3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2335371 INFO  (coreLoadExecutor-11743-thread-1-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw c:collection1   x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1563098904851906560
   [junit4]   2> 2335376 INFO  (coreZkRegister-11738-thread-1-processing-n:127.0.0.1:58206_co_lw x:collection1 c:collection1) [n:127.0.0.1:58206_co_lw c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2335376 INFO  (coreZkRegister-11738-thread-1-processing-n:127.0.0.1:58206_co_lw x:collection1 c:collection1) [n:127.0.0.1:58206_co_lw c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2335376 INFO  (coreZkRegister-11738-thread-1-processing-n:127.0.0.1:58206_co_lw x:collection1 c:collection1) [n:127.0.0.1:58206_co_lw c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:58206/co_lw/collection1/
   [junit4]   2> 2335376 INFO  (coreZkRegister-11738-thread-1-processing-n:127.0.0.1:58206_co_lw x:collection1 c:collection1) [n:127.0.0.1:58206_co_lw c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2335376 INFO  (coreZkRegister-11738-thread-1-processing-n:127.0.0.1:58206_co_lw x:collection1 c:collection1) [n:127.0.0.1:58206_co_lw c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:58206/co_lw/collection1/ has no replicas
   [junit4]   2> 2335376 INFO  (coreZkRegister-11738-thread-1-processing-n:127.0.0.1:58206_co_lw x:collection1 c:collection1) [n:127.0.0.1:58206_co_lw c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2335379 INFO  (coreZkRegister-11738-thread-1-processing-n:127.0.0.1:58206_co_lw x:collection1 c:collection1) [n:127.0.0.1:58206_co_lw c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:58206/co_lw/collection1/ shard1
   [junit4]   2> 2335482 INFO  (zkCallback-33225-thread-2-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2335530 INFO  (coreZkRegister-11738-thread-1-processing-n:127.0.0.1:58206_co_lw x:collection1 c:collection1) [n:127.0.0.1:58206_co_lw c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2335539 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-2-001/cores/collection1
   [junit4]   2> 2335540 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-2-001
   [junit4]   2> 2335540 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2335541 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e78acbc{/co_lw,null,AVAILABLE}
   [junit4]   2> 2335541 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5074f2d5{HTTP/1.1,[http/1.1]}{127.0.0.1:46815}
   [junit4]   2> 2335541 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.Server Started @2338491ms
   [junit4]   2> 2335541 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/co_lw, hostPort=46815, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-2-001/cores}
   [junit4]   2> 2335541 ERROR (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2335542 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 2335542 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2335542 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2335542 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-03-28T07:47:59.731Z
   [junit4]   2> 2335544 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2335544 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-2-001/solr.xml
   [junit4]   2> 2335550 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33250/solr
   [junit4]   2> 2335555 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:46815_co_lw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2335556 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:46815_co_lw    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2335557 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:46815_co_lw    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46815_co_lw
   [junit4]   2> 2335557 INFO  (zkCallback-33225-thread-2-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2335557 INFO  (zkCallback-33215-thread-3-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2335557 INFO  (zkCallback-33219-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2335558 INFO  (zkCallback-33225-thread-2-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2335584 INFO  (zkCallback-33231-thread-2-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2335630 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:46815_co_lw    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-2-001/cores
   [junit4]   2> 2335630 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:46815_co_lw    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 2335649 INFO  (OverseerStateUpdate-97693681418436612-127.0.0.1:52852_co_lw-n_0000000000) [n:127.0.0.1:52852_co_lw    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2335750 WARN  (OverseerStateUpdate-97693681418436612-127.0.0.1:52852_co_lw-n_0000000000) [n:127.0.0.1:52852_co_lw    ] o.a.s.c.Overseer Bad version writing to ZK using compare-and-set, will force refresh cluster state: KeeperErrorCode = BadVersion for /collections/collection1/state.json
   [junit4]   2> 2335752 INFO  (zkCallback-33225-thread-2-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2335752 INFO  (zkCallback-33231-thread-2-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2335753 INFO  (OverseerStateUpdate-97693681418436612-127.0.0.1:52852_co_lw-n_0000000000) [n:127.0.0.1:52852_co_lw    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2335753 INFO  (zkCallback-33225-thread-2-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2335753 INFO  (zkCallback-33231-thread-2-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2336661 WARN  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2336662 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2336675 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2336757 WARN  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 2336759 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 2336772 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 2336772 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2336772 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-2-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-2-001/cores/collection1/data/]
   [junit4]   2> 2336772 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c6651ed
   [junit4]   2> 2336774 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=50, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=2.126953125, noCFSRatio=0.13662606967834395]
   [junit4]   2> 2336783 WARN  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2336815 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2336815 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2336816 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
   [junit4]   2> 2336816 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2336817 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=27, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.0009765625, noCFSRatio=0.0]
   [junit4]   2> 2336817 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@22c4cba2[collection1] main]
   [junit4]   2> 2336818 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2336819 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2336819 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2336820 INFO  (searcherExecutor-11755-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 c:collection1) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@22c4cba2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2336820 INFO  (coreLoadExecutor-11754-thread-1-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw c:collection1   x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1563098906371293184
   [junit4]   2> 2336824 INFO  (coreZkRegister-11749-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 c:collection1) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
   [junit4]   2> 2336824 INFO  (updateExecutor-33228-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 2336824 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 2336825 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 2336825 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
   [junit4]   2> 2336825 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 2336825 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:58206/co_lw/collection1/] and I am [http://127.0.0.1:46815/co_lw/collection1/]
   [junit4]   2> 2336827 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:58206/co_lw]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:46815_co_lw&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 2336829 INFO  (qtp648844367-133339) [n:127.0.0.1:58206_co_lw    ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
   [junit4]   2> 2336830 INFO  (qtp648844367-133339) [n:127.0.0.1:58206_co_lw    ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
   [junit4]   2> 2336830 INFO  (qtp648844367-133339) [n:127.0.0.1:58206_co_lw    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:46815_co_lw, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"core":"collection1","base_url":"http://127.0.0.1:46815/co_lw","node_name":"127.0.0.1:46815_co_lw","state":"down"}
   [junit4]   2> 2336929 INFO  (zkCallback-33231-thread-2-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2336929 INFO  (zkCallback-33225-thread-2-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2336980 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-3-001/cores/collection1
   [junit4]   2> 2336981 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-3-001
   [junit4]   2> 2336981 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2336983 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2addaddc{/co_lw,null,AVAILABLE}
   [junit4]   2> 2336983 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@52cb905{HTTP/1.1,[http/1.1]}{127.0.0.1:48193}
   [junit4]   2> 2336983 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.Server Started @2339933ms
   [junit4]   2> 2336983 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/co_lw, hostPort=48193, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-3-001/cores}
   [junit4]   2> 2336983 ERROR (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2336984 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 2336984 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2336984 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2336984 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-03-28T07:48:01.173Z
   [junit4]   2> 2336986 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2336986 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-3-001/solr.xml
   [junit4]   2> 2336992 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33250/solr
   [junit4]   2> 2337002 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:48193_co_lw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 2337004 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:48193_co_lw    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2337006 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:48193_co_lw    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48193_co_lw
   [junit4]   2> 2337007 INFO  (zkCallback-33231-thread-2-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2337007 INFO  (zkCallback-33225-thread-2-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2337008 INFO  (zkCallback-33219-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2337008 INFO  (zkCallback-33215-thread-2-processing-n:127.0.0.1:52852_co_lw) [n:127.0.0.1:52852_co_lw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2337010 INFO  (zkCallback-33238-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2337074 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:48193_co_lw    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-3-001/cores
   [junit4]   2> 2337074 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [n:127.0.0.1:48193_co_lw    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 2337076 INFO  (OverseerStateUpdate-97693681418436612-127.0.0.1:52852_co_lw-n_0000000000) [n:127.0.0.1:52852_co_lw    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2337178 INFO  (zkCallback-33231-thread-2-processing-n:127.0.0.1:46815_co_lw) [n:127.0.0.1:46815_co_lw    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2337178 INFO  (zkCallback-33225-thread-2-processing-n:127.0.0.1:58206_co_lw) [n:127.0.0.1:58206_co_lw    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2337180 INFO  (zkCallback-33238-thread-2-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2337830 INFO  (qtp648844367-133339) [n:127.0.0.1:58206_co_lw    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:46815_co_lw, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"core":"collection1","base_url":"http://127.0.0.1:46815/co_lw","node_name":"127.0.0.1:46815_co_lw","state":"recovering"}
   [junit4]   2> 2337830 INFO  (qtp648844367-133339) [n:127.0.0.1:58206_co_lw    ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
   [junit4]   2> 2337830 INFO  (qtp648844367-133339) [n:127.0.0.1:58206_co_lw    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:46815_co_lw&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node2&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1000
   [junit4]   2> 2338089 WARN  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2338089 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2338102 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2338186 WARN  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 2338187 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 2338198 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 2338198 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2338198 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-3-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-3-001/cores/collection1/data/]
   [junit4]   2> 2338199 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c6651ed
   [junit4]   2> 2338200 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=50, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=2.126953125, noCFSRatio=0.13662606967834395]
   [junit4]   2> 2338211 WARN  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2338244 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2338244 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2338245 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
   [junit4]   2> 2338245 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2338246 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=27, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.0009765625, noCFSRatio=0.0]
   [junit4]   2> 2338246 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@ce482cd[collection1] main]
   [junit4]   2> 2338247 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2338247 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2338247 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2338248 INFO  (coreLoadExecutor-11765-thread-1-processing-n:127.0.0.1:48193_co_lw) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1563098907868659712
   [junit4]   2> 2338251 INFO  (searcherExecutor-11766-thread-1-processing-n:127.0.0.1:48193_co_lw x:collection1 c:collection1) [n:127.0.0.1:48193_co_lw c:collection1   x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@ce482cd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2338252 INFO  (coreZkRegister-11760-thread-1-processing-n:127.0.0.1:48193_co_lw x:collection1 c:collection1) [n:127.0.0.1:48193_co_lw c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
   [junit4]   2> 2338253 INFO  (updateExecutor-33235-thread-1-processing-n:127.0.0.1:48193_co_lw x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:48193_co_lw c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 2338253 INFO  (recoveryExecutor-33236-thread-1-processing-n:127.0.0.1:48193_co_lw x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:48193_co_lw c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 2338253 INFO  (recoveryExecutor-33236-thread-1-processing-n:127.0.0.1:48193_co_lw x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:48193_co_lw c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 2338253 INFO  (recoveryExecutor-33236-thread-1-processing-n:127.0.0.1:48193_co_lw x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:48193_co_lw c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
   [junit4]   2> 2338253 INFO  (recoveryExecutor-33236-thread-1-processing-n:127.0.0.1:48193_co_lw x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:48193_co_lw c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 2338253 INFO  (recoveryExecutor-33236-thread-1-processing-n:127.0.0.1:48193_co_lw x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:48193_co_lw c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:58206/co_lw/collection1/] and I am [http://127.0.0.1:48193/co_lw/collection1/]
   [junit4]   2> 2338256 INFO  (recoveryExecutor-33236-thread-1-processing-n:127.0.0.1:48193_co_lw x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:48193_co_lw c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:58206/co_lw]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:48193_co_lw&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 2338258 INFO  (qtp648844367-133343) [n:127.0.0.1:58206_co_lw    ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
   [junit4]   2> 2338258 INFO  (qtp648844367-133343) [n:127.0.0.1:58206_co_lw    ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
   [junit4]   2> 2338258 INFO  (qtp648844367-133343) [n:127.0.0.1:58206_co_lw    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:48193_co_lw, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:48193/co_lw","node_name":"127.0.0.1:48193_co_lw","state":"down"}
   [junit4]   2> 2338331 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Starting Replication Recovery.
   [junit4]   2> 2338331 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Attempting to replicate from [http://127.0.0.1:58206/co_lw/collection1/].
   [junit4]   2> 2338332 INFO  (qtp648844367-133341) [n:127.0.0.1:58206_co_lw c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1563098907956740096,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2338332 INFO  (qtp648844367-133341) [n:127.0.0.1:58206_co_lw c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 2338333 INFO  (qtp648844367-133341) [n:127.0.0.1:58206_co_lw c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2338333 INFO  (qtp648844367-133341) [n:127.0.0.1:58206_co_lw c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/co_lw path=/update params={waitSearcher=true&openSearcher=true&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2}{commit=} 0 0
   [junit4]   2> 2338334 INFO  (qtp648844367-133336) [n:127.0.0.1:58206_co_lw c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1]  webapp=/co_lw path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 2338334 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 2338334 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 2338334 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 2338334 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 2338334 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Replication Recovery was successful.
   [junit4]   2> 2338334 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
   [junit4]   2> 2338335 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 7000ms
   [junit4]   2> 2338335 WARN  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.h.ReplicationHandler SolrCloud is enabled for core collection1 but so is old-style replication. Make sure you intend this behavior, it usually indicates a mis-configuration. Master setting is false and slave setting is true
   [junit4]   2> 2338336 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Updating version bucket highest from index after successful recovery.
   [junit4]   2> 2338336 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1563098907960934400
   [junit4]   2> 2338338 INFO  (recoveryExecutor-33229-thread-1-processing-n:127.0.0.1:46815_co_lw x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:46815_co_lw c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[true]
   [junit4]   2> 2338408 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-4-001/cores/collection1
   [junit4]   2> 2338408 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-4-001
   [junit4]   2> 2338409 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2338410 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4590ce32{/co_lw,null,AVAILABLE}
   [junit4]   2> 2338410 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5f06ddc4{HTTP/1.1,[http/1.1]}{127.0.0.1:53910}
   [junit4]   2> 2338410 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.e.j.s.Server Started @2341360ms
   [junit4]   2> 2338410 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/tempDir-001/jetty4, solrconfig=solrconfig.xml, hostContext=/co_lw, hostPort=53910, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_D4517FC2A0E521C4-001/shard-4-001/cores}
   [junit4]   2> 2338410 ERROR (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2338411 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 2338411 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2338411 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2338411 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-03-28T07:48:02.600Z
   [junit4]   2> 2338413 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2338413 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[D4517FC2A0E521C4]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/so

[...truncated too long message...]

heckout/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/byte-buddy-1.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/mockito-core-2.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/test-lib/objenesis-2.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/analysis/icu/lucene-analyzers-icu-7.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_121/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.5.0.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J2-20170328_070901_1815764942187789337448.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J2-20170328_070901_1817138281819015634785.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1540)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
   [junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:783: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:719: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build.xml:267: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/common-build.xml:549: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/common-build.xml:1455: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/common-build.xml:985: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 627 minutes 4 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
ERROR: Failed to archive {checkout/solr/build/solr-core/test/temp/junit4-J2-20170328_070901_1815764942187789337448.events=checkout/solr/build/solr-core/test/temp/junit4-J2-20170328_070901_1815764942187789337448.events, checkout/solr/build/solr-core/test/temp/junit4-J0-20170328_070901_1716248727607123081291.events=checkout/solr/build/solr-core/test/temp/junit4-J0-20170328_070901_1716248727607123081291.events, checkout/README.txt=checkout/README.txt, checkout/solr/build/solr-core/test/temp/junit4-J1-20170328_070901_1714539054847433548477.events=checkout/solr/build/solr-core/test/temp/junit4-J1-20170328_070901_1714539054847433548477.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
        at jsync.protocol.ChangeInputStream.read(ChangeInputStream.java:69)
        at com.cloudbees.jenkins.plugins.jsync.archiver.MD5DigestingInputStream.read(MD5DigestingInputStream.java:39)
        at com.google.common.io.LimitInputStream.read(LimitInputStream.java:79)
        at java.io.FilterInputStream.read(FilterInputStream.java:107)
        at com.google.common.io.ByteStreams.copy(ByteStreams.java:193)
        at jsync.protocol.FileSequenceReader.read(FileSequenceReader.java:35)
        at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.remoteSync(JSyncArtifactManager.java:158)
        at com.cloudbees.jenkins.plugins.jsync.archiver.JSyncArtifactManager.archive(JSyncArtifactManager.java:76)
        at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:239)
        at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
        at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
        at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
        at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
        at hudson.model.Build$BuildExecution.post2(Build.java:185)
        at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:665)
        at hudson.model.Run.execute(Run.java:1753)
        at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
        at hudson.model.ResourceController.execute(ResourceController.java:98)
        at hudson.model.Executor.run(Executor.java:404)
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1276 - Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1276/

1 tests failed.
FAILED:  org.apache.solr.cloud.MissingSegmentRecoveryTest.testLeaderRecovery

Error Message:
Expected a collection with one shard and two replicas null Last available state: DocCollection(MissingSegmentRecoveryTest//collections/MissingSegmentRecoveryTest/state.json/8)={   "replicationFactor":"2",   "shards":{"shard1":{       "range":"80000000-7fffffff",       "state":"active",       "replicas":{         "core_node1":{           "core":"MissingSegmentRecoveryTest_shard1_replica2",           "base_url":"http://127.0.0.1:39888/solr",           "node_name":"127.0.0.1:39888_solr",           "state":"active",           "leader":"true"},         "core_node2":{           "core":"MissingSegmentRecoveryTest_shard1_replica1",           "base_url":"http://127.0.0.1:42936/solr",           "node_name":"127.0.0.1:42936_solr",           "state":"down"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"1",   "autoAddReplicas":"false",   "realtimeReplicas":"-1"}

Stack Trace:
java.lang.AssertionError: Expected a collection with one shard and two replicas
null
Last available state: DocCollection(MissingSegmentRecoveryTest//collections/MissingSegmentRecoveryTest/state.json/8)={
  "replicationFactor":"2",
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{
        "core_node1":{
          "core":"MissingSegmentRecoveryTest_shard1_replica2",
          "base_url":"http://127.0.0.1:39888/solr",
          "node_name":"127.0.0.1:39888_solr",
          "state":"active",
          "leader":"true"},
        "core_node2":{
          "core":"MissingSegmentRecoveryTest_shard1_replica1",
          "base_url":"http://127.0.0.1:42936/solr",
          "node_name":"127.0.0.1:42936_solr",
          "state":"down"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "realtimeReplicas":"-1"}
        at __randomizedtesting.SeedInfo.seed([FC5DD8A3B5EF4583:AC0840A0ECCEF39E]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:265)
        at org.apache.solr.cloud.MissingSegmentRecoveryTest.testLeaderRecovery(MissingSegmentRecoveryTest.java:105)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 13141 lines...]
   [junit4] Suite: org.apache.solr.cloud.MissingSegmentRecoveryTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/init-core-data-001
   [junit4]   2> 3402266 WARN  (SUITE-MissingSegmentRecoveryTest-seed#[FC5DD8A3B5EF4583]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=12 numCloses=12
   [junit4]   2> 3402267 INFO  (SUITE-MissingSegmentRecoveryTest-seed#[FC5DD8A3B5EF4583]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields
   [junit4]   2> 3402268 INFO  (SUITE-MissingSegmentRecoveryTest-seed#[FC5DD8A3B5EF4583]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 3402269 INFO  (SUITE-MissingSegmentRecoveryTest-seed#[FC5DD8A3B5EF4583]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001
   [junit4]   2> 3402269 INFO  (SUITE-MissingSegmentRecoveryTest-seed#[FC5DD8A3B5EF4583]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3402269 INFO  (Thread-182596) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3402269 INFO  (Thread-182596) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3402369 INFO  (SUITE-MissingSegmentRecoveryTest-seed#[FC5DD8A3B5EF4583]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:53002
   [junit4]   2> 3402385 INFO  (jetty-launcher-2116-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 3402385 INFO  (jetty-launcher-2116-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 3402387 INFO  (jetty-launcher-2116-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@794719aa{/solr,null,AVAILABLE}
   [junit4]   2> 3402387 INFO  (jetty-launcher-2116-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@52c22bbc{HTTP/1.1,[http/1.1]}{127.0.0.1:39888}
   [junit4]   2> 3402387 INFO  (jetty-launcher-2116-thread-1) [    ] o.e.j.s.Server Started @3408263ms
   [junit4]   2> 3402387 INFO  (jetty-launcher-2116-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39888}
   [junit4]   2> 3402388 ERROR (jetty-launcher-2116-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3402388 INFO  (jetty-launcher-2116-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 3402388 INFO  (jetty-launcher-2116-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3402388 INFO  (jetty-launcher-2116-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3402388 INFO  (jetty-launcher-2116-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-04T19:33:23.549Z
   [junit4]   2> 3402390 INFO  (jetty-launcher-2116-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@355aa80c{/solr,null,AVAILABLE}
   [junit4]   2> 3402391 INFO  (jetty-launcher-2116-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5d9fac34{HTTP/1.1,[http/1.1]}{127.0.0.1:42936}
   [junit4]   2> 3402391 INFO  (jetty-launcher-2116-thread-2) [    ] o.e.j.s.Server Started @3408266ms
   [junit4]   2> 3402391 INFO  (jetty-launcher-2116-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42936}
   [junit4]   2> 3402391 ERROR (jetty-launcher-2116-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3402391 INFO  (jetty-launcher-2116-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 3402391 INFO  (jetty-launcher-2116-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3402391 INFO  (jetty-launcher-2116-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3402391 INFO  (jetty-launcher-2116-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-04T19:33:23.552Z
   [junit4]   2> 3402407 INFO  (jetty-launcher-2116-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3402408 INFO  (jetty-launcher-2116-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3402420 INFO  (jetty-launcher-2116-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53002/solr
   [junit4]   2> 3402425 INFO  (jetty-launcher-2116-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53002/solr
   [junit4]   2> 3402446 INFO  (jetty-launcher-2116-thread-2) [n:127.0.0.1:42936_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3402456 INFO  (jetty-launcher-2116-thread-2) [n:127.0.0.1:42936_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42936_solr
   [junit4]   2> 3402457 INFO  (jetty-launcher-2116-thread-2) [n:127.0.0.1:42936_solr    ] o.a.s.c.Overseer Overseer (id=97736091463254021-127.0.0.1:42936_solr-n_0000000000) starting
   [junit4]   2> 3402468 INFO  (jetty-launcher-2116-thread-2) [n:127.0.0.1:42936_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42936_solr
   [junit4]   2> 3402478 INFO  (zkCallback-2127-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3402486 INFO  (jetty-launcher-2116-thread-1) [n:127.0.0.1:39888_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3402488 INFO  (jetty-launcher-2116-thread-1) [n:127.0.0.1:39888_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3402490 INFO  (jetty-launcher-2116-thread-1) [n:127.0.0.1:39888_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39888_solr
   [junit4]   2> 3402493 INFO  (zkCallback-2127-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3402502 INFO  (zkCallback-2128-thread-1-processing-n:127.0.0.1:39888_solr) [n:127.0.0.1:39888_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3402553 INFO  (jetty-launcher-2116-thread-2) [n:127.0.0.1:42936_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node2/.
   [junit4]   2> 3402655 INFO  (jetty-launcher-2116-thread-1) [n:127.0.0.1:39888_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node1/.
   [junit4]   2> 3403347 INFO  (SUITE-MissingSegmentRecoveryTest-seed#[FC5DD8A3B5EF4583]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 3403350 INFO  (SUITE-MissingSegmentRecoveryTest-seed#[FC5DD8A3B5EF4583]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53002/solr ready
   [junit4]   2> 3403374 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLeaderRecovery
   [junit4]   2> 3403375 INFO  (qtp708911146-193283) [n:127.0.0.1:39888_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&maxShardsPerNode=1&name=MissingSegmentRecoveryTest&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 3403377 INFO  (OverseerThreadFactory-5500-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.CreateCollectionCmd Create collection MissingSegmentRecoveryTest
   [junit4]   2> 3403489 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=MissingSegmentRecoveryTest_shard1_replica2&action=CREATE&numShards=1&collection=MissingSegmentRecoveryTest&shard=shard1&wt=javabin&version=2
   [junit4]   2> 3403489 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=MissingSegmentRecoveryTest_shard1_replica1&action=CREATE&numShards=1&collection=MissingSegmentRecoveryTest&shard=shard1&wt=javabin&version=2
   [junit4]   2> 3403489 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3403489 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3403595 INFO  (zkCallback-2128-thread-1-processing-n:127.0.0.1:39888_solr) [n:127.0.0.1:39888_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3403595 INFO  (zkCallback-2127-thread-2-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3404509 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 3404538 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 3404548 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.s.IndexSchema [MissingSegmentRecoveryTest_shard1_replica1] Schema name=minimal
   [junit4]   2> 3404551 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 3404551 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'MissingSegmentRecoveryTest_shard1_replica1' using configuration from collection MissingSegmentRecoveryTest, trusted=true
   [junit4]   2> 3404552 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 3404552 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrCore [[MissingSegmentRecoveryTest_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node2/MissingSegmentRecoveryTest_shard1_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node2/./MissingSegmentRecoveryTest_shard1_replica1/data/]
   [junit4]   2> 3404554 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.s.IndexSchema [MissingSegmentRecoveryTest_shard1_replica2] Schema name=minimal
   [junit4]   2> 3404557 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 3404557 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.CoreContainer Creating SolrCore 'MissingSegmentRecoveryTest_shard1_replica2' using configuration from collection MissingSegmentRecoveryTest, trusted=true
   [junit4]   2> 3404557 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 3404557 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrCore [[MissingSegmentRecoveryTest_shard1_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node1/MissingSegmentRecoveryTest_shard1_replica2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node1/./MissingSegmentRecoveryTest_shard1_replica2/data/]
   [junit4]   2> 3404616 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 3404616 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 3404616 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3404616 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3404617 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3404617 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3404617 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3404617 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3404619 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@74066870[MissingSegmentRecoveryTest_shard1_replica2] main]
   [junit4]   2> 3404619 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@66f50b5a[MissingSegmentRecoveryTest_shard1_replica1] main]
   [junit4]   2> 3404620 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 3404621 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 3404621 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3404621 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 3404621 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1563777465877266432
   [junit4]   2> 3404621 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 3404621 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 3404622 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1563777465878315008
   [junit4]   2> 3404623 INFO  (searcherExecutor-5506-thread-1-processing-n:127.0.0.1:39888_solr x:MissingSegmentRecoveryTest_shard1_replica2 s:shard1 c:MissingSegmentRecoveryTest) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica2] Registered new searcher Searcher@74066870[MissingSegmentRecoveryTest_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3404626 INFO  (searcherExecutor-5505-thread-1-processing-n:127.0.0.1:42936_solr x:MissingSegmentRecoveryTest_shard1_replica1 s:shard1 c:MissingSegmentRecoveryTest) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica1] Registered new searcher Searcher@66f50b5a[MissingSegmentRecoveryTest_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3404627 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 3404730 INFO  (zkCallback-2127-thread-2-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3404730 INFO  (zkCallback-2128-thread-1-processing-n:127.0.0.1:39888_solr) [n:127.0.0.1:39888_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3405128 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3405128 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3405128 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39888/solr/MissingSegmentRecoveryTest_shard1_replica2/
   [junit4]   2> 3405128 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.PeerSync PeerSync: core=MissingSegmentRecoveryTest_shard1_replica2 url=http://127.0.0.1:39888/solr START replicas=[http://127.0.0.1:42936/solr/MissingSegmentRecoveryTest_shard1_replica1/] nUpdates=100
   [junit4]   2> 3405129 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.PeerSync PeerSync: core=MissingSegmentRecoveryTest_shard1_replica2 url=http://127.0.0.1:39888/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 3405131 INFO  (qtp39821837-193296) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.S.Request [MissingSegmentRecoveryTest_shard1_replica1]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 3405131 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 3405131 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 3405131 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 3405135 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39888/solr/MissingSegmentRecoveryTest_shard1_replica2/ shard1
   [junit4]   2> 3405237 INFO  (zkCallback-2128-thread-1-processing-n:127.0.0.1:39888_solr) [n:127.0.0.1:39888_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3405238 INFO  (zkCallback-2127-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3405286 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3405288 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=MissingSegmentRecoveryTest_shard1_replica2&action=CREATE&numShards=1&collection=MissingSegmentRecoveryTest&shard=shard1&wt=javabin&version=2} status=0 QTime=1799
   [junit4]   2> 3405390 INFO  (zkCallback-2128-thread-1-processing-n:127.0.0.1:39888_solr) [n:127.0.0.1:39888_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3405390 INFO  (zkCallback-2127-thread-2-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3405632 INFO  (qtp39821837-193291) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1  x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=MissingSegmentRecoveryTest_shard1_replica1&action=CREATE&numShards=1&collection=MissingSegmentRecoveryTest&shard=shard1&wt=javabin&version=2} status=0 QTime=2143
   [junit4]   2> 3405635 INFO  (qtp708911146-193283) [n:127.0.0.1:39888_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 3405734 INFO  (zkCallback-2128-thread-1-processing-n:127.0.0.1:39888_solr) [n:127.0.0.1:39888_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3405734 INFO  (zkCallback-2127-thread-2-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3406636 INFO  (qtp708911146-193283) [n:127.0.0.1:39888_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&maxShardsPerNode=1&name=MissingSegmentRecoveryTest&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=3260
   [junit4]   2> 3406652 INFO  (qtp39821837-193297) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [MissingSegmentRecoveryTest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:39888/solr/MissingSegmentRecoveryTest_shard1_replica2/&wt=javabin&version=2}{add=[0 (1563777467993292800), 1 (1563777467995389952), 2 (1563777467995389953), 3 (1563777467995389954), 4 (1563777467995389955), 5 (1563777467995389956), 6 (1563777467995389957), 7 (1563777467995389958), 8 (1563777467995389959), 9 (1563777467995389960)]} 0 10
   [junit4]   2> 3406653 INFO  (qtp708911146-193284) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [MissingSegmentRecoveryTest_shard1_replica2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[0 (1563777467993292800), 1 (1563777467995389952), 2 (1563777467995389953), 3 (1563777467995389954), 4 (1563777467995389955), 5 (1563777467995389956), 6 (1563777467995389957), 7 (1563777467995389958), 8 (1563777467995389959), 9 (1563777467995389960)]} 0 14
   [junit4]   2> 3406654 INFO  (qtp39821837-193298) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 3406654 INFO  (qtp39821837-193298) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2598334c commitCommandVersion:0
   [junit4]   2> 3406654 INFO  (qtp708911146-193285) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 3406654 INFO  (qtp708911146-193285) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6f12df90 commitCommandVersion:0
   [junit4]   2> 3406684 INFO  (qtp39821837-193298) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6e70b31c[MissingSegmentRecoveryTest_shard1_replica1] main]
   [junit4]   2> 3406684 INFO  (qtp39821837-193298) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 3406686 INFO  (searcherExecutor-5505-thread-1-processing-n:127.0.0.1:42936_solr x:MissingSegmentRecoveryTest_shard1_replica1 s:shard1 c:MissingSegmentRecoveryTest r:core_node2) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica1] Registered new searcher Searcher@6e70b31c[MissingSegmentRecoveryTest_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C10)))}
   [junit4]   2> 3406686 INFO  (qtp39821837-193298) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [MissingSegmentRecoveryTest_shard1_replica1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39888/solr/MissingSegmentRecoveryTest_shard1_replica2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 32
   [junit4]   2> 3406693 INFO  (qtp708911146-193285) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@608aaad5[MissingSegmentRecoveryTest_shard1_replica2] main]
   [junit4]   2> 3406694 INFO  (searcherExecutor-5506-thread-1-processing-n:127.0.0.1:39888_solr x:MissingSegmentRecoveryTest_shard1_replica2 s:shard1 c:MissingSegmentRecoveryTest r:core_node1) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica2] Registered new searcher Searcher@608aaad5[MissingSegmentRecoveryTest_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C10)))}
   [junit4]   2> 3406694 INFO  (qtp708911146-193285) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 3406694 INFO  (qtp708911146-193285) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [MissingSegmentRecoveryTest_shard1_replica2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39888/solr/MissingSegmentRecoveryTest_shard1_replica2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 39
   [junit4]   2> 3406694 INFO  (qtp708911146-193287) [n:127.0.0.1:39888_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [MissingSegmentRecoveryTest_shard1_replica2]  webapp=/solr path=/update params={_stateVer_=MissingSegmentRecoveryTest:5&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 41
   [junit4]   2> 3406697 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@5d9fac34{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 3406700 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1215212236
   [junit4]   2> 3406700 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 3406702 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 3406703 INFO  (coreCloseExecutor-5515-thread-1) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica1]  CLOSING SolrCore org.apache.solr.core.SolrCore@62811361
   [junit4]   2> 3406707 INFO  (coreCloseExecutor-5515-thread-1) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.MissingSegmentRecoveryTest.shard1.replica1, tag=1652626273
   [junit4]   2> 3406707 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.a.s.c.Overseer Overseer (id=97736091463254021-127.0.0.1:42936_solr-n_0000000000) closing
   [junit4]   2> 3406709 INFO  (OverseerStateUpdate-97736091463254021-127.0.0.1:42936_solr-n_0000000000) [n:127.0.0.1:42936_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42936_solr
   [junit4]   2> 3406714 INFO  (zkCallback-2128-thread-1-processing-n:127.0.0.1:39888_solr) [n:127.0.0.1:39888_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39888_solr
   [junit4]   2> 3406714 INFO  (zkCallback-2134-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 3406715 INFO  (zkCallback-2128-thread-2-processing-n:127.0.0.1:39888_solr) [n:127.0.0.1:39888_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 3406716 INFO  (zkCallback-2128-thread-1-processing-n:127.0.0.1:39888_solr) [n:127.0.0.1:39888_solr    ] o.a.s.c.Overseer Overseer (id=97736091463254022-127.0.0.1:39888_solr-n_0000000001) starting
   [junit4]   2> 3406727 INFO  (zkCallback-2128-thread-1-processing-n:127.0.0.1:39888_solr) [n:127.0.0.1:39888_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3408214 WARN  (zkCallback-2127-thread-2-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 3408216 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@355aa80c{/solr,null,UNAVAILABLE}
   [junit4]   2> 3408217 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 3408218 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@25f26fc1{/solr,null,AVAILABLE}
   [junit4]   2> 3408218 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@593a07b{HTTP/1.1,[http/1.1]}{127.0.0.1:42936}
   [junit4]   2> 3408219 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.e.j.s.Server Started @3414094ms
   [junit4]   2> 3408219 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42936}
   [junit4]   2> 3408219 ERROR (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3408222 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 3408222 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3408222 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3408222 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-04T19:33:29.383Z
   [junit4]   2> 3408236 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3408246 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53002/solr
   [junit4]   2> 3408255 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [n:127.0.0.1:42936_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3408257 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [n:127.0.0.1:42936_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3408259 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [n:127.0.0.1:42936_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42936_solr
   [junit4]   2> 3408261 INFO  (zkCallback-2128-thread-2-processing-n:127.0.0.1:39888_solr) [n:127.0.0.1:39888_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3408271 INFO  (zkCallback-2139-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3408262 INFO  (zkCallback-2134-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3408261 INFO  (zkCallback-2128-thread-1-processing-n:127.0.0.1:39888_solr) [n:127.0.0.1:39888_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3408331 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [n:127.0.0.1:42936_solr    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node2/.
   [junit4]   2> 3408331 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [n:127.0.0.1:42936_solr    ] o.a.s.c.CorePropertiesLocator Cores are: [MissingSegmentRecoveryTest_shard1_replica1]
   [junit4]   2> 3408342 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3408355 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 3408363 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.s.IndexSchema [MissingSegmentRecoveryTest_shard1_replica1] Schema name=minimal
   [junit4]   2> 3408366 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 3408366 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'MissingSegmentRecoveryTest_shard1_replica1' using configuration from collection MissingSegmentRecoveryTest, trusted=true
   [junit4]   2> 3408366 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 3408366 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrCore [[MissingSegmentRecoveryTest_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node2/MissingSegmentRecoveryTest_shard1_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node2/./MissingSegmentRecoveryTest_shard1_replica1/data/]
   [junit4]   2> 3408429 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 3408429 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3408430 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3408430 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3408433 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica1]  CLOSING SolrCore org.apache.solr.core.SolrCore@5c41402f
   [junit4]   2> 3408433 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.MissingSegmentRecoveryTest.shard1.replica1, tag=1547780143
   [junit4]   2> 3408435 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.CoreContainer Found active leader, will attempt to create fresh core and recover.
   [junit4]   2> 3408435 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrCore Updating index properties... index=index.20170404223329596
   [junit4]   2> 3408439 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 3408439 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrCore [[MissingSegmentRecoveryTest_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node2/MissingSegmentRecoveryTest_shard1_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node2/./MissingSegmentRecoveryTest_shard1_replica1/data/]
   [junit4]   2> 3408446 INFO  (zkCallback-2139-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3408446 INFO  (zkCallback-2128-thread-1-processing-n:127.0.0.1:39888_solr) [n:127.0.0.1:39888_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3408454 INFO  (OldIndexDirectoryCleanupThreadForCore-MissingSegmentRecoveryTest_shard1_replica1) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.DirectoryFactory Found 1 old index directories to clean-up under /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node2/./MissingSegmentRecoveryTest_shard1_replica1/data/ afterReload=false
   [junit4]   2> 3408466 INFO  (OldIndexDirectoryCleanupThreadForCore-MissingSegmentRecoveryTest_shard1_replica1) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.DirectoryFactory Deleted old index directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node2/./MissingSegmentRecoveryTest_shard1_replica1/data/index.20170404223329596
   [junit4]   2> 3408470 INFO  (OldIndexDirectoryCleanupThreadForCore-MissingSegmentRecoveryTest_shard1_replica1) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.DirectoryFactory Found 1 old index directories to clean-up under /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node2/./MissingSegmentRecoveryTest_shard1_replica1/data/ afterReload=false
   [junit4]   2> 3408478 INFO  (OldIndexDirectoryCleanupThreadForCore-MissingSegmentRecoveryTest_shard1_replica1) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.DirectoryFactory Deleted old index directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node2/./MissingSegmentRecoveryTest_shard1_replica1/data/index
   [junit4]   2> 3408503 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 3408503 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3408504 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3408504 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3408506 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4575ee77
   [junit4]   2> 3408506 INFO  (coreLoadExecutor-5524-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.MissingSegmentRecoveryTest.shard1.replica1, tag=1165356663
   [junit4]   2> 3408507 ERROR (coreContainerWorkExecutor-5520-thread-1-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.CoreContainer Error waiting for SolrCore to be created
   [junit4]   2> java.util.concurrent.ExecutionException: org.apache.solr.common.SolrException: Unable to create core [MissingSegmentRecoveryTest_shard1_replica1]
   [junit4]   2> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
   [junit4]   2> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
   [junit4]   2> at org.apache.solr.core.CoreContainer.lambda$load$6(CoreContainer.java:614)
   [junit4]   2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Unable to create core [MissingSegmentRecoveryTest_shard1_replica1]
   [junit4]   2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
   [junit4]   2> at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:586)
   [junit4]   2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
   [junit4]   2> ... 5 more
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
   [junit4]   2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:985)
   [junit4]   2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:849)
   [junit4]   2> at org.apache.solr.core.CoreContainer.processCoreCreateException(CoreContainer.java:1038)
   [junit4]   2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:966)
   [junit4]   2> ... 7 more
   [junit4]   2> Suppressed: org.apache.solr.common.SolrException: Error opening new searcher
   [junit4]   2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:985)
   [junit4]   2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:849)
   [junit4]   2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:964)
   [junit4]   2> ... 7 more
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
   [junit4]   2> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2057)
   [junit4]   2> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:2177)
   [junit4]   2> at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:1074)
   [junit4]   2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:957)
   [junit4]   2> ... 9 more
   [junit4]   2> Caused by: org.apache.lucene.index.CorruptIndexException: Unexpected file read error while reading index. (resource=BufferedChecksumIndexInput(MMapIndexInput(path="/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node2/MissingSegmentRecoveryTest_shard1_replica1/data/index/segments_2")))
   [junit4]   2> at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:295)
   [junit4]   2> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:927)
   [junit4]   2> at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:119)
   [junit4]   2> at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:94)
   [junit4]   2> at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:248)
   [junit4]   2> at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:122)
   [junit4]   2> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2018)
   [junit4]   2> ... 12 more
   [junit4]   2> Caused by: java.io.EOFException: read past EOF: MMapIndexInput(path="/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node2/MissingSegmentRecoveryTest_shard1_replica1/data/index/segments_2")
   [junit4]   2> at org.apache.lucene.store.ByteBufferIndexInput.readByte(ByteBufferIndexInput.java:75)
   [junit4]   2> at org.apache.lucene.store.BufferedChecksumIndexInput.readByte(BufferedChecksumIndexInput.java:41)
   [junit4]   2> at org.apache.lucene.store.DataInput.readInt(DataInput.java:101)
   [junit4]   2> at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:305)
   [junit4]   2> at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:293)
   [junit4]   2> ... 18 more
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
   [junit4]   2> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2057)
   [junit4]   2> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:2177)
   [junit4]   2> at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:1074)
   [junit4]   2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:957)
   [junit4]   2> ... 10 more
   [junit4]   2> Caused by: org.apache.lucene.index.IndexNotFoundException: no segments* file found in LockValidatingDirectoryWrapper(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001/tempDir-001/node2/MissingSegmentRecoveryTest_shard1_replica1/data/index.20170404223329596 lockFactory=org.apache.lucene.store.NativeFSLockFactory@141edce8): files: [write.lock]
   [junit4]   2> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:922)
   [junit4]   2> at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:119)
   [junit4]   2> at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:94)
   [junit4]   2> at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:248)
   [junit4]   2> at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:122)
   [junit4]   2> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2018)
   [junit4]   2> ... 13 more
   [junit4]   2> 3498875 INFO  (qtp708911146-193280) [n:127.0.0.1:39888_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=MissingSegmentRecoveryTest&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 3498885 INFO  (OverseerThreadFactory-5518-thread-1-processing-n:127.0.0.1:39888_solr) [n:127.0.0.1:39888_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 3498886 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.MissingSegmentRecoveryTest.shard1.replica2, tag=null
   [junit4]   2> 3498886 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr    ] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica2]  CLOSING SolrCore org.apache.solr.core.SolrCore@44cb4a33
   [junit4]   2> 3498890 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.MissingSegmentRecoveryTest.shard1.replica2, tag=1154173491
   [junit4]   2> 3498900 INFO  (qtp896533997-193361) [n:127.0.0.1:42936_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=MissingSegmentRecoveryTest_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 3498901 INFO  (qtp708911146-193281) [n:127.0.0.1:39888_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=MissingSegmentRecoveryTest_shard1_replica2&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=15
   [junit4]   2> 3498904 INFO  (zkCallback-2139-thread-2-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3499009 INFO  (zkCallback-2139-thread-2-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 3499608 INFO  (qtp708911146-193280) [n:127.0.0.1:39888_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=MissingSegmentRecoveryTest&action=DELETE&wt=javabin&version=2} status=0 QTime=732
   [junit4]   2> 3499608 INFO  (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[FC5DD8A3B5EF4583]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLeaderRecovery
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=MissingSegmentRecoveryTest -Dtests.method=testLeaderRecovery -Dtests.seed=FC5DD8A3B5EF4583 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=ar-SY -Dtests.timezone=Europe/Nicosia -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 96.3s J2 | MissingSegmentRecoveryTest.testLeaderRecovery <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Expected a collection with one shard and two replicas
   [junit4]    > null
   [junit4]    > Last available state: DocCollection(MissingSegmentRecoveryTest//collections/MissingSegmentRecoveryTest/state.json/8)={
   [junit4]    >   "replicationFactor":"2",
   [junit4]    >   "shards":{"shard1":{
   [junit4]    >       "range":"80000000-7fffffff",
   [junit4]    >       "state":"active",
   [junit4]    >       "replicas":{
   [junit4]    >         "core_node1":{
   [junit4]    >           "core":"MissingSegmentRecoveryTest_shard1_replica2",
   [junit4]    >           "base_url":"http://127.0.0.1:39888/solr",
   [junit4]    >           "node_name":"127.0.0.1:39888_solr",
   [junit4]    >           "state":"active",
   [junit4]    >           "leader":"true"},
   [junit4]    >         "core_node2":{
   [junit4]    >           "core":"MissingSegmentRecoveryTest_shard1_replica1",
   [junit4]    >           "base_url":"http://127.0.0.1:42936/solr",
   [junit4]    >           "node_name":"127.0.0.1:42936_solr",
   [junit4]    >           "state":"down"}}}},
   [junit4]    >   "router":{"name":"compositeId"},
   [junit4]    >   "maxShardsPerNode":"1",
   [junit4]    >   "autoAddReplicas":"false",
   [junit4]    >   "realtimeReplicas":"-1"}
   [junit4]    > at __randomizedtesting.SeedInfo.seed([FC5DD8A3B5EF4583:AC0840A0ECCEF39E]:0)
   [junit4]    > at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:265)
   [junit4]    > at org.apache.solr.cloud.MissingSegmentRecoveryTest.testLeaderRecovery(MissingSegmentRecoveryTest.java:105)
   [junit4]    > at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3499613 INFO  (jetty-closer-2117-thread-2) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@52c22bbc{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 3499614 INFO  (jetty-closer-2117-thread-2) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=32718749
   [junit4]   2> 3499614 INFO  (jetty-closer-2117-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 3499615 INFO  (jetty-closer-2117-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 3499620 INFO  (jetty-closer-2117-thread-2) [    ] o.a.s.c.Overseer Overseer (id=97736091463254022-127.0.0.1:39888_solr-n_0000000001) closing
   [junit4]   2> 3499621 INFO  (OverseerStateUpdate-97736091463254022-127.0.0.1:39888_solr-n_0000000001) [n:127.0.0.1:39888_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:39888_solr
   [junit4]   2> 3499622 INFO  (zkCallback-2139-thread-2-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42936_solr
   [junit4]   2> 3499623 INFO  (zkCallback-2139-thread-2-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.Overseer Overseer (id=97736091463254027-127.0.0.1:42936_solr-n_0000000002) starting
   [junit4]   2> 3499626 INFO  (jetty-closer-2117-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@593a07b{HTTP/1.1,[http/1.1]}{127.0.0.1:42936}
   [junit4]   2> 3499627 INFO  (jetty-closer-2117-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=78540500
   [junit4]   2> 3499627 INFO  (jetty-closer-2117-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 3499627 INFO  (zkCallback-2139-thread-3-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 3499628 INFO  (jetty-closer-2117-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 3499641 INFO  (jetty-closer-2117-thread-1) [    ] o.a.s.c.Overseer Overseer (id=97736091463254027-127.0.0.1:42936_solr-n_0000000002) closing
   [junit4]   2> 3499641 INFO  (OverseerStateUpdate-97736091463254027-127.0.0.1:42936_solr-n_0000000002) [n:127.0.0.1:42936_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42936_solr
   [junit4]   2> 3499642 ERROR (OverseerCollectionConfigSetProcessor-97736091463254027-127.0.0.1:42936_solr-n_0000000002) [n:127.0.0.1:42936_solr    ] o.a.s.c.OverseerTaskProcessor Unable to prioritize overseer
   [junit4]   2> java.lang.InterruptedException
   [junit4]   2> at java.lang.Object.wait(Native Method)
   [junit4]   2> at java.lang.Object.wait(Object.java:502)
   [junit4]   2> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1342)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1040)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:322)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:319)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:319)
   [junit4]   2> at org.apache.solr.cloud.OverseerNodePrioritizer.prioritizeOverseerNodes(OverseerNodePrioritizer.java:61)
   [junit4]   2> at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:178)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3499654 WARN  (zkCallback-2139-thread-3-processing-n:127.0.0.1:42936_solr) [n:127.0.0.1:42936_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 3499655 INFO  (jetty-closer-2117-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@25f26fc1{/solr,null,UNAVAILABLE}
   [junit4]   2> 3501122 WARN  (zkCallback-2128-thread-19-processing-n:127.0.0.1:39888_solr) [n:127.0.0.1:39888_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 3501124 INFO  (jetty-closer-2117-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@794719aa{/solr,null,UNAVAILABLE}
   [junit4]   2> 3501125 INFO  (SUITE-MissingSegmentRecoveryTest-seed#[FC5DD8A3B5EF4583]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:53002 53002
   [junit4]   2> 3501224 INFO  (Thread-182596) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:53002 53002
   [junit4]   2> 3501224 WARN  (Thread-182596) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 4 /solr/aliases.json
   [junit4]   2> 4 /solr/clusterprops.json
   [junit4]   2> 3 /solr/security.json
   [junit4]   2> 2 /solr/configs/conf
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 4 /solr/clusterstate.json
   [junit4]   2> 4 /solr/collections/MissingSegmentRecoveryTest/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 94 /solr/overseer/collection-queue-work
   [junit4]   2> 13 /solr/overseer/queue
   [junit4]   2> 4 /solr/collections
   [junit4]   2> 4 /solr/overseer/queue-work
   [junit4]   2> 3 /solr/live_nodes
   [junit4]   2>
   [junit4]   2> 3501225 INFO  (SUITE-MissingSegmentRecoveryTest-seed#[FC5DD8A3B5EF4583]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MissingSegmentRecoveryTest_FC5DD8A3B5EF4583-001
   [junit4]   2> Apr 04, 2017 7:35:02 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=true): {}, locale=ar-SY, timezone=Europe/Nicosia
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=39343296,total=526385152
   [junit4]   2> NOTE: All tests run in this JVM: [TestJsonFacets, TestLegacyNumericRangeQueryBuilder, TestSolrDeletionPolicy1, HdfsBasicDistributedZkTest, JsonValidatorTest, TestInitParams, ReplicaListTransformerTest, SolrShardReporterTest, DistributedExpandComponentTest, JavabinLoaderTest, FieldAnalysisRequestHandlerTest, TestPhraseSuggestions, LukeRequestHandlerTest, TestSolrJ, TestSearcherReuse, TestStressReorder, XmlUpdateRequestHandlerTest, TestLMDirichletSimilarityFactory, DistributedSuggestComponentTest, HdfsWriteToMultipleCollectionsTest, TestExclusionRuleCollectionAccess, TestCoreDiscovery, TestCustomDocTransformer, TestSubQueryTransformerDistrib, TestIBSimilarityFactory, BasicAuthIntegrationTest, TestReload, TestClusterProperties, ZkStateReaderTest, DebugComponentTest, ShowFileRequestHandlerTest, FullSolrCloudDistribCmdsTest, TestTolerantSearch, TestSchemaManager, TestCustomSort, JSONWriterTest, EchoParamsTest, TestHdfsUpdateLog, StatsReloadRaceTest, TestSlowCompositeReaderWrapper, ReplicationFactorTest, ConjunctionSolrSpellCheckerTest, OverseerCollectionConfigSetProcessorTest, TestRawResponseWriter, TestExtendedDismaxParser, TestGraphTermsQParserPlugin, TestLeaderElectionZkExpiry, CheckHdfsIndexTest, CdcrVersionReplicationTest, TestScoreJoinQPNoScore, DirectUpdateHandlerOptimizeTest, HdfsDirectoryTest, TestDeleteCollectionOnDownNodes, SolrJmxReporterTest, TestReversedWildcardFilterFactory, SolrIndexMetricsTest, TestZkChroot, TestSegmentSorting, TestSolrCloudWithDelegationTokens, SolrMetricsIntegrationTest, TestExactSharedStatsCache, RecoveryAfterSoftCommitTest, HdfsCollectionsAPIDistributedZkTest, BadCopyFieldTest, PrimUtilsTest, OverseerStatusTest, TestUninvertingReader, CloudMLTQParserTest, HighlighterConfigTest, TestFieldTypeCollectionResource, TestCollapseQParserPlugin, TestConfigOverlay, TestDistributedMissingSort, BadIndexSchemaTest, SolrCoreCheckLockOnStartupTest, HLLUtilTest, SolrIndexConfigTest, TestComponentsName, TestMultiWordSynonyms, OverseerTest, TermVectorComponentDistributedTest, UpdateParamsTest, SuggesterWFSTTest, OutputWriterTest, TestMiniSolrCloudClusterSSL, TestBinaryField, ClassificationUpdateProcessorTest, ConcurrentDeleteAndCreateCollectionTest, TestAuthorizationFramework, TestPHPSerializedResponseWriter, TestSolrCLIRunExample, PrimitiveFieldTypeTest, CurrencyFieldXmlFileTest, CursorMarkTest, TestStandardQParsers, RuleEngineTest, TestSolrQueryParser, CollectionTooManyReplicasTest, HardAutoCommitTest, RankQueryTest, TestXmlQParser, TestRestManager, TestConfig, SimpleFacetsTest, MBeansHandlerTest, TestQueryWrapperFilter, ActionThrottleTest, URLClassifyProcessorTest, TestMergePolicyConfig, TestCloudPivotFacet, TestManagedSchemaAPI, TestRestoreCore, RecoveryZkTest, LeaderElectionTest, TestDistributedSearch, DeleteShardTest, DistribCursorPagingTest, DistributedVersionInfoTest, LeaderElectionContextKeyTest, LeaderFailureAfterFreshStartTest, MigrateRouteKeyTest, MissingSegmentRecoveryTest]
   [junit4] Completed [499/701 (1!)] on J2 in 99.75s, 1 test, 1 failure <<< FAILURES!

[...truncated 43920 lines...]




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1277 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1277/

1 tests failed.
FAILED:  org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv

Error Message:
java.lang.RuntimeException: Error from server at http://127.0.0.1:37196/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:46611/solr/test_col_shard5_replica1: Server Error    request: http://127.0.0.1:46611/solr/test_col_shard5_replica1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A37196%2Fsolr%2Ftest_col_shard3_replica1%2F&wt=javabin&version=2 Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:33659/solr/test_col_shard5_replica2/ update: org.apache.solr.client.solrj.request.UpdateRequest@20e4ddcf

Stack Trace:
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error from server at http://127.0.0.1:37196/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:46611/solr/test_col_shard5_replica1: Server Error



request: http://127.0.0.1:46611/solr/test_col_shard5_replica1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A37196%2Fsolr%2Ftest_col_shard3_replica1%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:33659/solr/test_col_shard5_replica2/ update: org.apache.solr.client.solrj.request.UpdateRequest@20e4ddcf
        at __randomizedtesting.SeedInfo.seed([21B1B767741BFF2:340F7930FD1C85E3]:0)
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:281)
        at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:193)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Error from server at http://127.0.0.1:37196/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:46611/solr/test_col_shard5_replica1: Server Error



request: http://127.0.0.1:46611/solr/test_col_shard5_replica1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A37196%2Fsolr%2Ftest_col_shard3_replica1%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:33659/solr/test_col_shard5_replica2/ update: org.apache.solr.client.solrj.request.UpdateRequest@20e4ddcf
        at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:409)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        ... 1 more
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37196/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:46611/solr/test_col_shard5_replica1: Server Error



request: http://127.0.0.1:46611/solr/test_col_shard5_replica1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A37196%2Fsolr%2Ftest_col_shard3_replica1%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:33659/solr/test_col_shard5_replica2/ update: org.apache.solr.client.solrj.request.UpdateRequest@20e4ddcf
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
        at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.doRandomAtomicUpdate(TestStressCloudBlindAtomicUpdates.java:368)
        at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:397)
        ... 6 more




Build Log:
[...truncated 12418 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/init-core-data-001
   [junit4]   2> 2199489 WARN  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[21B1B767741BFF2]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 2199490 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[21B1B767741BFF2]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields
   [junit4]   2> 2199492 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[21B1B767741BFF2]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=SSL overhead seems to cause OutOfMemory when stress testing)
   [junit4]   2> 2199500 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[21B1B767741BFF2]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 10 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001
   [junit4]   2> 2199500 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[21B1B767741BFF2]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2199506 INFO  (Thread-33801) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2199506 INFO  (Thread-33801) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2199606 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[21B1B767741BFF2]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:44900
   [junit4]   2> 2199627 INFO  (jetty-launcher-1429-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2199627 INFO  (jetty-launcher-1429-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2199634 INFO  (jetty-launcher-1429-thread-3) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2199635 INFO  (jetty-launcher-1429-thread-4) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2199657 INFO  (jetty-launcher-1429-thread-5) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2199657 INFO  (jetty-launcher-1429-thread-6) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2199658 INFO  (jetty-launcher-1429-thread-7) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2199663 INFO  (jetty-launcher-1429-thread-9) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2199664 INFO  (jetty-launcher-1429-thread-7) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5810158{/solr,null,AVAILABLE}
   [junit4]   2> 2199666 INFO  (jetty-launcher-1429-thread-7) [    ] o.e.j.s.AbstractConnector Started ServerConnector@18be00aa{HTTP/1.1,[http/1.1]}{127.0.0.1:33659}
   [junit4]   2> 2199666 INFO  (jetty-launcher-1429-thread-7) [    ] o.e.j.s.Server Started @2207066ms
   [junit4]   2> 2199666 INFO  (jetty-launcher-1429-thread-7) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33659}
   [junit4]   2> 2199666 ERROR (jetty-launcher-1429-thread-7) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2199667 INFO  (jetty-launcher-1429-thread-7) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2199667 INFO  (jetty-launcher-1429-thread-7) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2199667 INFO  (jetty-launcher-1429-thread-7) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2199667 INFO  (jetty-launcher-1429-thread-7) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-07T14:09:55.011Z
   [junit4]   2> 2199668 INFO  (jetty-launcher-1429-thread-6) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6a80360c{/solr,null,AVAILABLE}
   [junit4]   2> 2199669 INFO  (jetty-launcher-1429-thread-6) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1f360096{HTTP/1.1,[http/1.1]}{127.0.0.1:44751}
   [junit4]   2> 2199669 INFO  (jetty-launcher-1429-thread-6) [    ] o.e.j.s.Server Started @2207068ms
   [junit4]   2> 2199669 INFO  (jetty-launcher-1429-thread-6) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44751}
   [junit4]   2> 2199669 ERROR (jetty-launcher-1429-thread-6) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2199669 INFO  (jetty-launcher-1429-thread-6) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2199669 INFO  (jetty-launcher-1429-thread-6) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2199669 INFO  (jetty-launcher-1429-thread-6) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2199669 INFO  (jetty-launcher-1429-thread-6) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-07T14:09:55.013Z
   [junit4]   2> 2199677 INFO  (jetty-launcher-1429-thread-6) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2199686 INFO  (jetty-launcher-1429-thread-9) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62a505a4{/solr,null,AVAILABLE}
   [junit4]   2> 2199687 INFO  (jetty-launcher-1429-thread-10) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2199689 INFO  (jetty-launcher-1429-thread-8) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2199689 INFO  (jetty-launcher-1429-thread-9) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7977514a{HTTP/1.1,[http/1.1]}{127.0.0.1:41398}
   [junit4]   2> 2199689 INFO  (jetty-launcher-1429-thread-9) [    ] o.e.j.s.Server Started @2207089ms
   [junit4]   2> 2199690 INFO  (jetty-launcher-1429-thread-9) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41398}
   [junit4]   2> 2199690 ERROR (jetty-launcher-1429-thread-9) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2199690 INFO  (jetty-launcher-1429-thread-9) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2199690 INFO  (jetty-launcher-1429-thread-9) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2199690 INFO  (jetty-launcher-1429-thread-9) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2199690 INFO  (jetty-launcher-1429-thread-9) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-07T14:09:55.034Z
   [junit4]   2> 2199693 INFO  (jetty-launcher-1429-thread-10) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@33d39285{/solr,null,AVAILABLE}
   [junit4]   2> 2199694 INFO  (jetty-launcher-1429-thread-10) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4c8eae44{HTTP/1.1,[http/1.1]}{127.0.0.1:37196}
   [junit4]   2> 2199694 INFO  (jetty-launcher-1429-thread-10) [    ] o.e.j.s.Server Started @2207094ms
   [junit4]   2> 2199694 INFO  (jetty-launcher-1429-thread-10) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37196}
   [junit4]   2> 2199694 ERROR (jetty-launcher-1429-thread-10) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2199694 INFO  (jetty-launcher-1429-thread-10) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2199694 INFO  (jetty-launcher-1429-thread-10) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2199694 INFO  (jetty-launcher-1429-thread-10) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2199694 INFO  (jetty-launcher-1429-thread-10) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-07T14:09:55.038Z
   [junit4]   2> 2199795 INFO  (jetty-launcher-1429-thread-5) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@738c3c9{/solr,null,AVAILABLE}
   [junit4]   2> 2199796 INFO  (jetty-launcher-1429-thread-5) [    ] o.e.j.s.AbstractConnector Started ServerConnector@35b3fb29{HTTP/1.1,[http/1.1]}{127.0.0.1:59957}
   [junit4]   2> 2199796 INFO  (jetty-launcher-1429-thread-5) [    ] o.e.j.s.Server Started @2207196ms
   [junit4]   2> 2199796 INFO  (jetty-launcher-1429-thread-5) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59957}
   [junit4]   2> 2199796 ERROR (jetty-launcher-1429-thread-5) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2199796 INFO  (jetty-launcher-1429-thread-5) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2199796 INFO  (jetty-launcher-1429-thread-5) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2199796 INFO  (jetty-launcher-1429-thread-5) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2199796 INFO  (jetty-launcher-1429-thread-5) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-07T14:09:55.140Z
   [junit4]   2> 2199799 INFO  (jetty-launcher-1429-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@73ab2b56{/solr,null,AVAILABLE}
   [junit4]   2> 2199799 INFO  (jetty-launcher-1429-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@12e1080b{HTTP/1.1,[http/1.1]}{127.0.0.1:46611}
   [junit4]   2> 2199799 INFO  (jetty-launcher-1429-thread-2) [    ] o.e.j.s.Server Started @2207199ms
   [junit4]   2> 2199799 INFO  (jetty-launcher-1429-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46611}
   [junit4]   2> 2199800 ERROR (jetty-launcher-1429-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2199800 INFO  (jetty-launcher-1429-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2199800 INFO  (jetty-launcher-1429-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2199800 INFO  (jetty-launcher-1429-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2199800 INFO  (jetty-launcher-1429-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-07T14:09:55.144Z
   [junit4]   2> 2199816 INFO  (jetty-launcher-1429-thread-10) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2199827 INFO  (jetty-launcher-1429-thread-7) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2199844 INFO  (jetty-launcher-1429-thread-6) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44900/solr
   [junit4]   2> 2199845 INFO  (jetty-launcher-1429-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@38450000{/solr,null,AVAILABLE}
   [junit4]   2> 2199850 INFO  (jetty-launcher-1429-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1b4d091a{/solr,null,AVAILABLE}
   [junit4]   2> 2199848 INFO  (jetty-launcher-1429-thread-5) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2199847 INFO  (jetty-launcher-1429-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2199846 INFO  (jetty-launcher-1429-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@26023047{/solr,null,AVAILABLE}
   [junit4]   2> 2199866 INFO  (jetty-launcher-1429-thread-8) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@16cf49e1{/solr,null,AVAILABLE}
   [junit4]   2> 2199866 INFO  (jetty-launcher-1429-thread-8) [    ] o.e.j.s.AbstractConnector Started ServerConnector@29e1b094{HTTP/1.1,[http/1.1]}{127.0.0.1:40738}
   [junit4]   2> 2199866 INFO  (jetty-launcher-1429-thread-8) [    ] o.e.j.s.Server Started @2207266ms
   [junit4]   2> 2199866 INFO  (jetty-launcher-1429-thread-8) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40738}
   [junit4]   2> 2199867 ERROR (jetty-launcher-1429-thread-8) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2199867 INFO  (jetty-launcher-1429-thread-8) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2199867 INFO  (jetty-launcher-1429-thread-8) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2199867 INFO  (jetty-launcher-1429-thread-8) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2199867 INFO  (jetty-launcher-1429-thread-8) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-07T14:09:55.211Z
   [junit4]   2> 2199869 INFO  (jetty-launcher-1429-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4355af4{HTTP/1.1,[http/1.1]}{127.0.0.1:55816}
   [junit4]   2> 2199869 INFO  (jetty-launcher-1429-thread-3) [    ] o.e.j.s.Server Started @2207269ms
   [junit4]   2> 2199869 INFO  (jetty-launcher-1429-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55816}
   [junit4]   2> 2199869 ERROR (jetty-launcher-1429-thread-3) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2199876 INFO  (jetty-launcher-1429-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2199876 INFO  (jetty-launcher-1429-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2199876 INFO  (jetty-launcher-1429-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2199876 INFO  (jetty-launcher-1429-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-07T14:09:55.220Z
   [junit4]   2> 2199876 INFO  (jetty-launcher-1429-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@31a56c92{HTTP/1.1,[http/1.1]}{127.0.0.1:59985}
   [junit4]   2> 2199877 INFO  (jetty-launcher-1429-thread-4) [    ] o.e.j.s.Server Started @2207276ms
   [junit4]   2> 2199877 INFO  (jetty-launcher-1429-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59985}
   [junit4]   2> 2199877 ERROR (jetty-launcher-1429-thread-4) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2199877 INFO  (jetty-launcher-1429-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2199877 INFO  (jetty-launcher-1429-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2199877 INFO  (jetty-launcher-1429-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2199877 INFO  (jetty-launcher-1429-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-07T14:09:55.221Z
   [junit4]   2> 2199892 INFO  (jetty-launcher-1429-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@368fe8ed{HTTP/1.1,[http/1.1]}{127.0.0.1:41211}
   [junit4]   2> 2199892 INFO  (jetty-launcher-1429-thread-1) [    ] o.e.j.s.Server Started @2207292ms
   [junit4]   2> 2199892 INFO  (jetty-launcher-1429-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41211}
   [junit4]   2> 2199892 ERROR (jetty-launcher-1429-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2199894 INFO  (jetty-launcher-1429-thread-9) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2199894 INFO  (jetty-launcher-1429-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2199894 INFO  (jetty-launcher-1429-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2199894 INFO  (jetty-launcher-1429-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2199894 INFO  (jetty-launcher-1429-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-07T14:09:55.238Z
   [junit4]   2> 2199900 INFO  (jetty-launcher-1429-thread-8) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2199901 INFO  (jetty-launcher-1429-thread-7) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44900/solr
   [junit4]   2> 2199903 INFO  (jetty-launcher-1429-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44900/solr
   [junit4]   2> 2199911 INFO  (jetty-launcher-1429-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2199929 INFO  (jetty-launcher-1429-thread-10) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44900/solr
   [junit4]   2> 2199930 INFO  (jetty-launcher-1429-thread-5) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44900/solr
   [junit4]   2> 2199905 INFO  (jetty-launcher-1429-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2199959 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b48c002a9000e, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2199961 INFO  (jetty-launcher-1429-thread-9) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44900/solr
   [junit4]   2> 2199961 INFO  (jetty-launcher-1429-thread-8) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44900/solr
   [junit4]   2> 2199920 INFO  (jetty-launcher-1429-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2199966 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b48c002a9000a, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2199991 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b48c002a9000c, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2199996 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b48c002a90010, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2200007 INFO  (jetty-launcher-1429-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44900/solr
   [junit4]   2> 2200008 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b48c002a90009, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2200013 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b48c002a90013, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2200013 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b48c002a90011, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2200028 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b48c002a90014, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2200030 INFO  (jetty-launcher-1429-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44900/solr
   [junit4]   2> 2200036 INFO  (jetty-launcher-1429-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44900/solr
   [junit4]   2> 2200051 INFO  (jetty-launcher-1429-thread-7) [n:127.0.0.1:33659_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2200052 INFO  (jetty-launcher-1429-thread-6) [n:127.0.0.1:44751_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2200062 INFO  (jetty-launcher-1429-thread-7) [n:127.0.0.1:33659_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33659_solr
   [junit4]   2> 2200068 INFO  (jetty-launcher-1429-thread-6) [n:127.0.0.1:44751_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44751_solr
   [junit4]   2> 2200069 INFO  (jetty-launcher-1429-thread-7) [n:127.0.0.1:33659_solr    ] o.a.s.c.Overseer Overseer (id=97751806434148367-127.0.0.1:33659_solr-n_0000000000) starting
   [junit4]   2> 2200081 INFO  (jetty-launcher-1429-thread-2) [n:127.0.0.1:46611_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2200082 INFO  (jetty-launcher-1429-thread-8) [n:127.0.0.1:40738_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2200082 INFO  (jetty-launcher-1429-thread-10) [n:127.0.0.1:37196_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2200090 INFO  (jetty-launcher-1429-thread-2) [n:127.0.0.1:46611_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2200091 INFO  (jetty-launcher-1429-thread-8) [n:127.0.0.1:40738_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2200091 INFO  (jetty-launcher-1429-thread-10) [n:127.0.0.1:37196_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2200095 INFO  (jetty-launcher-1429-thread-8) [n:127.0.0.1:40738_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40738_solr
   [junit4]   2> 2200095 INFO  (jetty-launcher-1429-thread-10) [n:127.0.0.1:37196_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37196_solr
   [junit4]   2> 2200139 INFO  (jetty-launcher-1429-thread-9) [n:127.0.0.1:41398_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 2200140 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b48c002a90017, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2200142 INFO  (jetty-launcher-1429-thread-9) [n:127.0.0.1:41398_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2200143 INFO  (jetty-launcher-1429-thread-9) [n:127.0.0.1:41398_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41398_solr
   [junit4]   2> 2200186 INFO  (jetty-launcher-1429-thread-2) [n:127.0.0.1:46611_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46611_solr
   [junit4]   2> 2200193 INFO  (jetty-launcher-1429-thread-5) [n:127.0.0.1:59957_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
   [junit4]   2> 2200195 INFO  (jetty-launcher-1429-thread-5) [n:127.0.0.1:59957_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2200319 INFO  (zkCallback-1460-thread-1-processing-n:127.0.0.1:33659_solr) [n:127.0.0.1:33659_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 2200319 INFO  (zkCallback-1456-thread-1-processing-n:127.0.0.1:44751_solr) [n:127.0.0.1:44751_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 2200319 INFO  (zkCallback-1464-thread-1-processing-n:127.0.0.1:46611_solr) [n:127.0.0.1:46611_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (4)
   [junit4]   2> 2200319 INFO  (zkCallback-1472-thread-1-processing-n:127.0.0.1:40738_solr) [n:127.0.0.1:40738_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (4)
   [junit4]   2> 2200319 INFO  (zkCallback-1480-thread-1-processing-n:127.0.0.1:41398_solr) [n:127.0.0.1:41398_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2200319 INFO  (zkCallback-1468-thread-1-processing-n:127.0.0.1:37196_solr) [n:127.0.0.1:37196_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (4)
   [junit4]   2> 2200327 INFO  (jetty-launcher-1429-thread-5) [n:127.0.0.1:59957_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59957_solr
   [junit4]   2> 2200387 INFO  (zkCallback-1460-thread-2-processing-n:127.0.0.1:33659_solr) [n:127.0.0.1:33659_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (6)
   [junit4]   2> 2200395 INFO  (zkCallback-1464-thread-2-processing-n:127.0.0.1:46611_solr) [n:127.0.0.1:46611_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (6)
   [junit4]   2> 2200406 INFO  (zkCallback-1472-thread-2-processing-n:127.0.0.1:40738_solr) [n:127.0.0.1:40738_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (6)
   [junit4]   2> 2200419 INFO  (zkCallback-1468-thread-2-processing-n:127.0.0.1:37196_solr) [n:127.0.0.1:37196_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (6)
   [junit4]   2> 2200436 INFO  (zkCallback-1456-thread-2-processing-n:127.0.0.1:44751_solr) [n:127.0.0.1:44751_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (6)
   [junit4]   2> 2200455 INFO  (jetty-launcher-1429-thread-1) [n:127.0.0.1:41211_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (6)
   [junit4]   2> 2200457 INFO  (jetty-launcher-1429-thread-1) [n:127.0.0.1:41211_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2200459 INFO  (zkCallback-1480-thread-2-processing-n:127.0.0.1:41398_solr) [n:127.0.0.1:41398_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (6)
   [junit4]   2> 2200459 INFO  (jetty-launcher-1429-thread-1) [n:127.0.0.1:41211_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41211_solr
   [junit4]   2> 2200464 INFO  (jetty-launcher-1429-thread-10) [n:127.0.0.1:37196_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node10/.
   [junit4]   2> 2200497 INFO  (OverseerStateUpdate-97751806434148367-127.0.0.1:33659_solr-n_0000000000) [n:127.0.0.1:33659_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
   [junit4]   2> 2200539 INFO  (jetty-launcher-1429-thread-7) [n:127.0.0.1:33659_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33659_solr
   [junit4]   2> 2200560 INFO  (jetty-launcher-1429-thread-4) [n:127.0.0.1:59985_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (8)
   [junit4]   2> 2200562 INFO  (jetty-launcher-1429-thread-4) [n:127.0.0.1:59985_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2200564 INFO  (jetty-launcher-1429-thread-4) [n:127.0.0.1:59985_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59985_solr
   [junit4]   2> 2200593 INFO  (zkCallback-1473-thread-1-processing-n:127.0.0.1:59957_solr) [n:127.0.0.1:59957_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (7)
   [junit4]   2> 2200602 INFO  (jetty-launcher-1429-thread-3) [n:127.0.0.1:55816_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (9)
   [junit4]   2> 2200604 INFO  (jetty-launcher-1429-thread-3) [n:127.0.0.1:55816_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2200606 INFO  (jetty-launcher-1429-thread-3) [n:127.0.0.1:55816_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55816_solr
   [junit4]   2> 2200623 INFO  (zkCallback-1460-thread-2-processing-n:127.0.0.1:33659_solr) [n:127.0.0.1:33659_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (10)
   [junit4]   2> 2200624 INFO  (zkCallback-1456-thread-2-processing-n:127.0.0.1:44751_solr) [n:127.0.0.1:44751_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (10)
   [junit4]   2> 2200624 INFO  (zkCallback-1472-thread-2-processing-n:127.0.0.1:40738_solr) [n:127.0.0.1:40738_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (10)
   [junit4]   2> 2200624 INFO  (zkCallback-1480-thread-2-processing-n:127.0.0.1:41398_solr) [n:127.0.0.1:41398_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (10)
   [junit4]   2> 2200631 INFO  (zkCallback-1464-thread-2-processing-n:127.0.0.1:46611_solr) [n:127.0.0.1:46611_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (10)
   [junit4]   2> 2200631 INFO  (zkCallback-1468-thread-2-processing-n:127.0.0.1:37196_solr) [n:127.0.0.1:37196_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (10)
   [junit4]   2> 2200689 INFO  (zkCallback-1485-thread-1-processing-n:127.0.0.1:41211_solr) [n:127.0.0.1:41211_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (10)
   [junit4]   2> 2200703 INFO  (zkCallback-1473-thread-2-processing-n:127.0.0.1:59957_solr) [n:127.0.0.1:59957_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (10)
   [junit4]   2> 2200919 INFO  (zkCallback-1486-thread-2-processing-n:127.0.0.1:55816_solr) [n:127.0.0.1:55816_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
   [junit4]   2> 2200919 INFO  (zkCallback-1487-thread-1-processing-n:127.0.0.1:59985_solr) [n:127.0.0.1:59985_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (10)
   [junit4]   2> 2200944 INFO  (jetty-launcher-1429-thread-5) [n:127.0.0.1:59957_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node5/.
   [junit4]   2> 2200945 INFO  (jetty-launcher-1429-thread-9) [n:127.0.0.1:41398_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node9/.
   [junit4]   2> 2200954 INFO  (jetty-launcher-1429-thread-6) [n:127.0.0.1:44751_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node6/.
   [junit4]   2> 2200973 INFO  (jetty-launcher-1429-thread-1) [n:127.0.0.1:41211_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node1/.
   [junit4]   2> 2200975 INFO  (jetty-launcher-1429-thread-2) [n:127.0.0.1:46611_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node2/.
   [junit4]   2> 2201017 INFO  (jetty-launcher-1429-thread-4) [n:127.0.0.1:59985_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node4/.
   [junit4]   2> 2201028 INFO  (jetty-launcher-1429-thread-8) [n:127.0.0.1:40738_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node8/.
   [junit4]   2> 2201091 INFO  (jetty-launcher-1429-thread-3) [n:127.0.0.1:55816_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node3/.
   [junit4]   2> 2201317 INFO  (jetty-launcher-1429-thread-7) [n:127.0.0.1:33659_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node7/.
   [junit4]   2> 2203772 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[21B1B767741BFF2]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (10)
   [junit4]   2> 2203773 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[21B1B767741BFF2]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44900/solr ready
   [junit4]   2> 2204214 INFO  (qtp2073345578-41179) [n:127.0.0.1:55816_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&name=test_col&action=CREATE&numShards=5&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2204289 INFO  (OverseerThreadFactory-3538-thread-1-processing-n:127.0.0.1:33659_solr) [n:127.0.0.1:33659_solr    ] o.a.s.c.CreateCollectionCmd Create collection test_col
   [junit4]   2> 2204403 INFO  (qtp2073345578-41136) [n:127.0.0.1:55816_solr    ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&name=test_col_shard2_replica1&action=CREATE&numShards=5&collection=test_col&shard=shard2&wt=javabin&version=2
   [junit4]   2> 2204403 INFO  (qtp2073345578-41136) [n:127.0.0.1:55816_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2204419 INFO  (qtp10115430-41131) [n:127.0.0.1:59957_solr    ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&name=test_col_shard4_replica1&action=CREATE&numShards=5&collection=test_col&shard=shard4&wt=javabin&version=2
   [junit4]   2> 2204419 INFO  (qtp2134374752-41188) [n:127.0.0.1:40738_solr    ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&name=test_col_shard3_replica2&action=CREATE&numShards=5&collection=test_col&shard=shard3&wt=javabin&version=2
   [junit4]   2> 2204419 INFO  (qtp2134374752-41188) [n:127.0.0.1:40738_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2204419 INFO  (qtp585236496-41152) [n:127.0.0.1:37196_solr    ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&name=test_col_shard3_replica1&action=CREATE&numShards=5&collection=test_col&shard=shard3&wt=javabin&version=2
   [junit4]   2> 2204419 INFO  (qtp1550421315-41180) [n:127.0.0.1:59985_solr    ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&name=test_col_shard1_replica1&action=CREATE&numShards=5&collection=test_col&shard=shard1&wt=javabin&version=2
   [junit4]   2> 2204420 INFO  (qtp585236496-41152) [n:127.0.0.1:37196_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2204420 INFO  (qtp1550421315-41180) [n:127.0.0.1:59985_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2204420 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr    ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&name=test_col_shard1_replica2&action=CREATE&numShards=5&collection=test_col&shard=shard1&wt=javabin&version=2
   [junit4]   2> 2204420 INFO  (qtp1353913597-41169) [n:127.0.0.1:46611_solr    ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&name=test_col_shard5_replica1&action=CREATE&numShards=5&collection=test_col&shard=shard5&wt=javabin&version=2
   [junit4]   2> 2204420 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2204420 INFO  (qtp1353913597-41169) [n:127.0.0.1:46611_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2204420 INFO  (qtp1780983665-41210) [n:127.0.0.1:41211_solr    ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&name=test_col_shard2_replica2&action=CREATE&numShards=5&collection=test_col&shard=shard2&wt=javabin&version=2
   [junit4]   2> 2204421 INFO  (qtp1780983665-41210) [n:127.0.0.1:41211_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2204421 INFO  (qtp1000444165-41143) [n:127.0.0.1:41398_solr    ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&name=test_col_shard4_replica2&action=CREATE&numShards=5&collection=test_col&shard=shard4&wt=javabin&version=2
   [junit4]   2> 2204422 INFO  (qtp10115430-41131) [n:127.0.0.1:59957_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2204422 INFO  (qtp1000444165-41143) [n:127.0.0.1:41398_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2204422 INFO  (qtp2065413026-41109) [n:127.0.0.1:33659_solr    ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&name=test_col_shard5_replica2&action=CREATE&numShards=5&collection=test_col&shard=shard5&wt=javabin&version=2
   [junit4]   2> 2204423 INFO  (qtp2065413026-41109) [n:127.0.0.1:33659_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2204543 INFO  (zkCallback-1472-thread-2-processing-n:127.0.0.1:40738_solr) [n:127.0.0.1:40738_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
   [junit4]   2> 2204543 INFO  (zkCallback-1473-thread-2-processing-n:127.0.0.1:59957_solr) [n:127.0.0.1:59957_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
   [junit4]   2> 2204543 INFO  (zkCallback-1468-thread-2-processing-n:127.0.0.1:37196_solr) [n:127.0.0.1:37196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
   [junit4]   2> 2204544 INFO  (zkCallback-1486-thread-2-processing-n:127.0.0.1:55816_solr) [n:127.0.0.1:55816_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
   [junit4]   2> 2204544 INFO  (zkCallback-1485-thread-1-processing-n:127.0.0.1:41211_solr) [n:127.0.0.1:41211_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
   [junit4]   2> 2204544 INFO  (zkCallback-1480-thread-2-processing-n:127.0.0.1:41398_solr) [n:127.0.0.1:41398_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
   [junit4]   2> 2204544 INFO  (zkCallback-1456-thread-2-processing-n:127.0.0.1:44751_solr) [n:127.0.0.1:44751_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
   [junit4]   2> 2204545 INFO  (zkCallback-1464-thread-2-processing-n:127.0.0.1:46611_solr) [n:127.0.0.1:46611_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
   [junit4]   2> 2204545 INFO  (zkCallback-1487-thread-1-processing-n:127.0.0.1:59985_solr) [n:127.0.0.1:59985_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
   [junit4]   2> 2204551 INFO  (zkCallback-1460-thread-2-processing-n:127.0.0.1:33659_solr) [n:127.0.0.1:33659_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
   [junit4]   2> 2205426 WARN  (qtp2073345578-41136) [n:127.0.0.1:55816_solr c:test_col s:shard2  x:test_col_shard2_replica1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2205443 INFO  (qtp2073345578-41136) [n:127.0.0.1:55816_solr c:test_col s:shard2  x:test_col_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2205469 WARN  (qtp1780983665-41210) [n:127.0.0.1:41211_solr c:test_col s:shard2  x:test_col_shard2_replica2] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2205470 INFO  (qtp1780983665-41210) [n:127.0.0.1:41211_solr c:test_col s:shard2  x:test_col_shard2_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2205496 WARN  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2205497 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2205500 WARN  (qtp2134374752-41188) [n:127.0.0.1:40738_solr c:test_col s:shard3  x:test_col_shard3_replica2] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2205501 INFO  (qtp2134374752-41188) [n:127.0.0.1:40738_solr c:test_col s:shard3  x:test_col_shard3_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2205512 INFO  (qtp2073345578-41136) [n:127.0.0.1:55816_solr c:test_col s:shard2  x:test_col_shard2_replica1] o.a.s.s.IndexSchema [test_col_shard2_replica1] Schema name=minimal-atomic-stress
   [junit4]   2> 2205514 WARN  (qtp10115430-41131) [n:127.0.0.1:59957_solr c:test_col s:shard4  x:test_col_shard4_replica1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2205518 WARN  (qtp585236496-41152) [n:127.0.0.1:37196_solr c:test_col s:shard3  x:test_col_shard3_replica1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2205519 INFO  (qtp585236496-41152) [n:127.0.0.1:37196_solr c:test_col s:shard3  x:test_col_shard3_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2205522 WARN  (qtp1550421315-41180) [n:127.0.0.1:59985_solr c:test_col s:shard1  x:test_col_shard1_replica1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2205524 WARN  (qtp2065413026-41109) [n:127.0.0.1:33659_solr c:test_col s:shard5  x:test_col_shard5_replica2] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2205525 INFO  (qtp2065413026-41109) [n:127.0.0.1:33659_solr c:test_col s:shard5  x:test_col_shard5_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2205526 INFO  (qtp2073345578-41136) [n:127.0.0.1:55816_solr c:test_col s:shard2  x:test_col_shard2_replica1] o.a.s.s.IndexSchema Loaded schema minimal-atomic-stress/1.6 with uniqueid field id
   [junit4]   2> 2205526 INFO  (qtp2073345578-41136) [n:127.0.0.1:55816_solr c:test_col s:shard2  x:test_col_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard2_replica1' using configuration from collection test_col, trusted=true
   [junit4]   2> 2205526 INFO  (qtp2073345578-41136) [n:127.0.0.1:55816_solr c:test_col s:shard2  x:test_col_shard2_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2205526 INFO  (qtp2073345578-41136) [n:127.0.0.1:55816_solr c:test_col s:shard2  x:test_col_shard2_replica1] o.a.s.c.SolrCore [[test_col_shard2_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node3/test_col_shard2_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node3/./test_col_shard2_replica1/data/]
   [junit4]   2> 2205527 INFO  (qtp2073345578-41136) [n:127.0.0.1:55816_solr c:test_col s:shard2  x:test_col_shard2_replica1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4af3d8d7
   [junit4]   2> 2205528 WARN  (qtp1353913597-41169) [n:127.0.0.1:46611_solr c:test_col s:shard5  x:test_col_shard5_replica1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2205529 INFO  (qtp1353913597-41169) [n:127.0.0.1:46611_solr c:test_col s:shard5  x:test_col_shard5_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2205530 WARN  (qtp1000444165-41143) [n:127.0.0.1:41398_solr c:test_col s:shard4  x:test_col_shard4_replica2] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2205531 INFO  (qtp1000444165-41143) [n:127.0.0.1:41398_solr c:test_col s:shard4  x:test_col_shard4_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2205534 INFO  (qtp1780983665-41210) [n:127.0.0.1:41211_solr c:test_col s:shard2  x:test_col_shard2_replica2] o.a.s.s.IndexSchema [test_col_shard2_replica2] Schema name=minimal-atomic-stress
   [junit4]   2> 2205535 INFO  (qtp10115430-41131) [n:127.0.0.1:59957_solr c:test_col s:shard4  x:test_col_shard4_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2205536 INFO  (qtp2073345578-41136) [n:127.0.0.1:55816_solr c:test_col s:shard2  x:test_col_shard2_replica1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=91.3720703125, floorSegmentMB=1.716796875, forceMergeDeletesPctAllowed=14.886592209738957, segmentsPerTier=39.0, maxCFSSegmentSizeMB=0.236328125, noCFSRatio=0.0
   [junit4]   2> 2205538 INFO  (qtp2134374752-41188) [n:127.0.0.1:40738_solr c:test_col s:shard3  x:test_col_shard3_replica2] o.a.s.s.IndexSchema [test_col_shard3_replica2] Schema name=minimal-atomic-stress
   [junit4]   2> 2205573 INFO  (qtp1550421315-41180) [n:127.0.0.1:59985_solr c:test_col s:shard1  x:test_col_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2205574 INFO  (qtp1780983665-41210) [n:127.0.0.1:41211_solr c:test_col s:shard2  x:test_col_shard2_replica2] o.a.s.s.IndexSchema Loaded schema minimal-atomic-stress/1.6 with uniqueid field id
   [junit4]   2> 2205574 INFO  (qtp1780983665-41210) [n:127.0.0.1:41211_solr c:test_col s:shard2  x:test_col_shard2_replica2] o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard2_replica2' using configuration from collection test_col, trusted=true
   [junit4]   2> 2205574 INFO  (qtp1780983665-41210) [n:127.0.0.1:41211_solr c:test_col s:shard2  x:test_col_shard2_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2205574 INFO  (qtp1780983665-41210) [n:127.0.0.1:41211_solr c:test_col s:shard2  x:test_col_shard2_replica2] o.a.s.c.SolrCore [[test_col_shard2_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node1/test_col_shard2_replica2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node1/./test_col_shard2_replica2/data/]
   [junit4]   2> 2205575 INFO  (qtp1780983665-41210) [n:127.0.0.1:41211_solr c:test_col s:shard2  x:test_col_shard2_replica2] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4af3d8d7
   [junit4]   2> 2205576 INFO  (qtp2134374752-41188) [n:127.0.0.1:40738_solr c:test_col s:shard3  x:test_col_shard3_replica2] o.a.s.s.IndexSchema Loaded schema minimal-atomic-stress/1.6 with uniqueid field id
   [junit4]   2> 2205576 INFO  (qtp2134374752-41188) [n:127.0.0.1:40738_solr c:test_col s:shard3  x:test_col_shard3_replica2] o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard3_replica2' using configuration from collection test_col, trusted=true
   [junit4]   2> 2205577 INFO  (qtp2134374752-41188) [n:127.0.0.1:40738_solr c:test_col s:shard3  x:test_col_shard3_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2205577 INFO  (qtp2134374752-41188) [n:127.0.0.1:40738_solr c:test_col s:shard3  x:test_col_shard3_replica2] o.a.s.c.SolrCore [[test_col_shard3_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node8/test_col_shard3_replica2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node8/./test_col_shard3_replica2/data/]
   [junit4]   2> 2205577 INFO  (qtp2134374752-41188) [n:127.0.0.1:40738_solr c:test_col s:shard3  x:test_col_shard3_replica2] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4af3d8d7
   [junit4]   2> 2205579 INFO  (qtp1780983665-41210) [n:127.0.0.1:41211_solr c:test_col s:shard2  x:test_col_shard2_replica2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=91.3720703125, floorSegmentMB=1.716796875, forceMergeDeletesPctAllowed=14.886592209738957, segmentsPerTier=39.0, maxCFSSegmentSizeMB=0.236328125, noCFSRatio=0.0
   [junit4]   2> 2205579 INFO  (qtp2134374752-41188) [n:127.0.0.1:40738_solr c:test_col s:shard3  x:test_col_shard3_replica2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=91.3720703125, floorSegmentMB=1.716796875, forceMergeDeletesPctAllowed=14.886592209738957, segmentsPerTier=39.0, maxCFSSegmentSizeMB=0.236328125, noCFSRatio=0.0
   [junit4]   2> 2205585 INFO  (qtp10115430-41131) [n:127.0.0.1:59957_solr c:test_col s:shard4  x:test_col_shard4_replica1] o.a.s.s.IndexSchema [test_col_shard4_replica1] Schema name=minimal-atomic-stress
   [junit4]   2> 2205585 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.s.IndexSchema [test_col_shard1_replica2] Schema name=minimal-atomic-stress
   [junit4]   2> 2205589 INFO  (qtp1000444165-41143) [n:127.0.0.1:41398_solr c:test_col s:shard4  x:test_col_shard4_replica2] o.a.s.s.IndexSchema [test_col_shard4_replica2] Schema name=minimal-atomic-stress
   [junit4]   2> 2205591 INFO  (qtp10115430-41131) [n:127.0.0.1:59957_solr c:test_col s:shard4  x:test_col_shard4_replica1] o.a.s.s.IndexSchema Loaded schema minimal-atomic-stress/1.6 with uniqueid field id
   [junit4]   2> 2205591 INFO  (qtp10115430-41131) [n:127.0.0.1:59957_solr c:test_col s:shard4  x:test_col_shard4_replica1] o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard4_replica1' using configuration from collection test_col, trusted=true
   [junit4]   2> 2205592 INFO  (qtp10115430-41131) [n:127.0.0.1:59957_solr c:test_col s:shard4  x:test_col_shard4_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2205592 INFO  (qtp10115430-41131) [n:127.0.0.1:59957_solr c:test_col s:shard4  x:test_col_shard4_replica1] o.a.s.c.SolrCore [[test_col_shard4_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node5/test_col_shard4_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node5/./test_col_shard4_replica1/data/]
   [junit4]   2> 2205592 INFO  (qtp10115430-41131) [n:127.0.0.1:59957_solr c:test_col s:shard4  x:test_col_shard4_replica1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4af3d8d7
   [junit4]   2> 2205593 INFO  (qtp1000444165-41143) [n:127.0.0.1:41398_solr c:test_col s:shard4  x:test_col_shard4_replica2] o.a.s.s.IndexSchema Loaded schema minimal-atomic-stress/1.6 with uniqueid field id
   [junit4]   2> 2205593 INFO  (qtp1000444165-41143) [n:127.0.0.1:41398_solr c:test_col s:shard4  x:test_col_shard4_replica2] o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard4_replica2' using configuration from collection test_col, trusted=true
   [junit4]   2> 2205593 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.s.IndexSchema Loaded schema minimal-atomic-stress/1.6 with uniqueid field id
   [junit4]   2> 2205593 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard1_replica2' using configuration from collection test_col, trusted=true
   [junit4]   2> 2205594 INFO  (qtp1000444165-41143) [n:127.0.0.1:41398_solr c:test_col s:shard4  x:test_col_shard4_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2205594 INFO  (qtp1000444165-41143) [n:127.0.0.1:41398_solr c:test_col s:shard4  x:test_col_shard4_replica2] o.a.s.c.SolrCore [[test_col_shard4_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node9/test_col_shard4_replica2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node9/./test_col_shard4_replica2/data/]
   [junit4]   2> 2205594 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2205594 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.c.SolrCore [[test_col_shard1_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node6/test_col_shard1_replica2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node6/./test_col_shard1_replica2/data/]
   [junit4]   2> 2205594 INFO  (qtp1000444165-41143) [n:127.0.0.1:41398_solr c:test_col s:shard4  x:test_col_shard4_replica2] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4af3d8d7
   [junit4]   2> 2205594 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4af3d8d7
   [junit4]   2> 2205599 WARN  (qtp1780983665-41210) [n:127.0.0.1:41211_solr c:test_col s:shard2  x:test_col_shard2_replica2] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2205600 INFO  (qtp1000444165-41143) [n:127.0.0.1:41398_solr c:test_col s:shard4  x:test_col_shard4_replica2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=91.3720703125, floorSegmentMB=1.716796875, forceMergeDeletesPctAllowed=14.886592209738957, segmentsPerTier=39.0, maxCFSSegmentSizeMB=0.236328125, noCFSRatio=0.0
   [junit4]   2> 2205599 INFO  (qtp10115430-41131) [n:127.0.0.1:59957_solr c:test_col s:shard4  x:test_col_shard4_replica1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=91.3720703125, floorSegmentMB=1.716796875, forceMergeDeletesPctAllowed=14.886592209738957, segmentsPerTier=39.0, maxCFSSegmentSizeMB=0.236328125, noCFSRatio=0.0
   [junit4]   2> 2205602 INFO  (qtp585236496-41152) [n:127.0.0.1:37196_solr c:test_col s:shard3  x:test_col_shard3_replica1] o.a.s.s.IndexSchema [test_col_shard3_replica1] Schema name=minimal-atomic-stress
   [junit4]   2> 2205608 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=91.3720703125, floorSegmentMB=1.716796875, forceMergeDeletesPctAllowed=14.886592209738957, segmentsPerTier=39.0, maxCFSSegmentSizeMB=0.236328125, noCFSRatio=0.0
   [junit4]   2> 2205612 WARN  (qtp2134374752-41188) [n:127.0.0.1:40738_solr c:test_col s:shard3  x:test_col_shard3_replica2] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2205613 INFO  (qtp585236496-41152) [n:127.0.0.1:37196_solr c:test_col s:shard3  x:test_col_shard3_replica1] o.a.s.s.IndexSchema Loaded schema minimal-atomic-stress/1.6 with uniqueid field id
   [junit4]   2> 2205613 INFO  (qtp585236496-41152) [n:127.0.0.1:37196_solr c:test_col s:shard3  x:test_col_shard3_replica1] o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard3_replica1' using configuration from collection test_col, trusted=true
   [junit4]   2> 2205613 INFO  (qtp585236496-41152) [n:127.0.0.1:37196_solr c:test_col s:shard3  x:test_col_shard3_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2205613 INFO  (qtp585236496-41152) [n:127.0.0.1:37196_solr c:test_col s:shard3  x:test_col_shard3_replica1] o.a.s.c.SolrCore [[test_col_shard3_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node10/test_col_shard3_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node10/./test_col_shard3_replica1/data/]
   [junit4]   2> 2205614 INFO  (qtp585236496-41152) [n:127.0.0.1:37196_solr c:test_col s:shard3  x:test_col_shard3_replica1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4af3d8d7
   [junit4]   2> 2205616 INFO  (qtp1353913597-41169) [n:127.0.0.1:46611_solr c:test_col s:shard5  x:test_col_shard5_replica1] o.a.s.s.IndexSchema [test_col_shard5_replica1] Schema name=minimal-atomic-stress
   [junit4]   2> 2205619 WARN  (qtp2073345578-41136) [n:127.0.0.1:55816_solr c:test_col s:shard2  x:test_col_shard2_replica1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2205622 INFO  (qtp585236496-41152) [n:127.0.0.1:37196_solr c:test_col s:shard3  x:test_col_shard3_replica1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=91.3720703125, floorSegmentMB=1.716796875, forceMergeDeletesPctAllowed=14.886592209738957, segmentsPerTier=39.0, maxCFSSegmentSizeMB=0.236328125, noCFSRatio=0.0
   [junit4]   2> 2205622 INFO  (qtp1353913597-41169) [n:127.0.0.1:46611_solr c:test_col s:shard5  x:test_col_shard5_replica1] o.a.s.s.IndexSchema Loaded schema minimal-atomic-stress/1.6 with uniqueid field id
   [junit4]   2> 2205622 INFO  (qtp1353913597-41169) [n:127.0.0.1:46611_solr c:test_col s:shard5  x:test_col_shard5_replica1] o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard5_replica1' using configuration from collection test_col, trusted=true
   [junit4]   2> 2205622 INFO  (qtp1353913597-41169) [n:127.0.0.1:46611_solr c:test_col s:shard5  x:test_col_shard5_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2205622 INFO  (qtp1353913597-41169) [n:127.0.0.1:46611_solr c:test_col s:shard5  x:test_col_shard5_replica1] o.a.s.c.SolrCore [[test_col_shard5_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node2/test_col_shard5_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node2/./test_col_shard5_replica1/data/]
   [junit4]   2> 2205623 INFO  (qtp1353913597-41169) [n:127.0.0.1:46611_solr c:test_col s:shard5  x:test_col_shard5_replica1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4af3d8d7
   [junit4]   2> 2205624 INFO  (qtp1353913597-41169) [n:127.0.0.1:46611_solr c:test_col s:shard5  x:test_col_shard5_replica1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=91.3720703125, floorSegmentMB=1.716796875, forceMergeDeletesPctAllowed=14.886592209738957, segmentsPerTier=39.0, maxCFSSegmentSizeMB=0.236328125, noCFSRatio=0.0
   [junit4]   2> 2205625 WARN  (qtp10115430-41131) [n:127.0.0.1:59957_solr c:test_col s:shard4  x:test_col_shard4_replica1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2205636 WARN  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2205641 WARN  (qtp1000444165-41143) [n:127.0.0.1:41398_solr c:test_col s:shard4  x:test_col_shard4_replica2] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2205669 WARN  (qtp585236496-41152) [n:127.0.0.1:37196_solr c:test_col s:shard3  x:test_col_shard3_replica1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2205673 INFO  (qtp1550421315-41180) [n:127.0.0.1:59985_solr c:test_col s:shard1  x:test_col_shard1_replica1] o.a.s.s.IndexSchema [test_col_shard1_replica1] Schema name=minimal-atomic-stress
   [junit4]   2> 2205675 INFO  (qtp1550421315-41180) [n:127.0.0.1:59985_solr c:test_col s:shard1  x:test_col_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal-atomic-stress/1.6 with uniqueid field id
   [junit4]   2> 2205675 INFO  (qtp1550421315-41180) [n:127.0.0.1:59985_solr c:test_col s:shard1  x:test_col_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard1_replica1' using configuration from collection test_col, trusted=true
   [junit4]   2> 2205676 INFO  (qtp1550421315-41180) [n:127.0.0.1:59985_solr c:test_col s:shard1  x:test_col_shard1_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2205676 INFO  (qtp1550421315-41180) [n:127.0.0.1:59985_solr c:test_col s:shard1  x:test_col_shard1_replica1] o.a.s.c.SolrCore [[test_col_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node4/test_col_shard1_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node4/./test_col_shard1_replica1/data/]
   [junit4]   2> 2205676 INFO  (qtp1550421315-41180) [n:127.0.0.1:59985_solr c:test_col s:shard1  x:test_col_shard1_replica1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4af3d8d7
   [junit4]   2> 2205678 INFO  (qtp1550421315-41180) [n:127.0.0.1:59985_solr c:test_col s:shard1  x:test_col_shard1_replica1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=91.3720703125, floorSegmentMB=1.716796875, forceMergeDeletesPctAllowed=14.886592209738957, segmentsPerTier=39.0, maxCFSSegmentSizeMB=0.236328125, noCFSRatio=0.0
   [junit4]   2> 2205692 INFO  (qtp2065413026-41109) [n:127.0.0.1:33659_solr c:test_col s:shard5  x:test_col_shard5_replica2] o.a.s.s.IndexSchema [test_col_shard5_replica2] Schema name=minimal-atomic-stress
   [junit4]   2> 2205694 INFO  (qtp2065413026-41109) [n:127.0.0.1:33659_solr c:test_col s:shard5  x:test_col_shard5_replica2] o.a.s.s.IndexSchema Loaded schema minimal-atomic-stress/1.6 with uniqueid field id
   [junit4]   2> 2205694 INFO  (qtp2065413026-41109) [n:127.0.0.1:33659_solr c:test_col s:shard5  x:test_col_shard5_replica2] o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard5_replica2' using configuration from collection test_col, trusted=true
   [junit4]   2> 2205696 WARN  (qtp1353913597-41169) [n:127.0.0.1:46611_solr c:test_col s:shard5  x:test_col_shard5_replica1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2205700 INFO  (qtp2065413026-41109) [n:127.0.0.1:33659_solr c:test_col s:shard5  x:test_col_shard5_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2205700 INFO  (qtp2065413026-41109) [n:127.0.0.1:33659_solr c:test_col s:shard5  x:test_col_shard5_replica2] o.a.s.c.SolrCore [[test_col_shard5_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node7/test_col_shard5_replica2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001/tempDir-001/node7/./test_col_shard5_replica2/data/]
   [junit4]   2> 2205701 WARN  (qtp1550421315-41180) [n:127.0.0.1:59985_solr c:test_col s:shard1  x:test_col_shard1_replica1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2205701 INFO  (qtp2065413026-41109) [n:127.0.0.1:33659_solr c:test_col s:shard5  x:test_col_shard5_replica2] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4af3d8d7
   [junit4]   2> 2205702 INFO  (qtp2065413026-41109) [n:127.0.0.1:33659_solr c:test_col s:shard5  x:test_col_shard5_replica2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=91.3720703125, floorSegmentMB=1.716796875, forceMergeDeletesPctAllowed=14.886592209738957, segmentsPerTier=39.0, maxCFSSegmentSizeMB=0.236328125, noCFSRatio=0.0
   [junit4]   2> 2205709 WARN  (qtp2065413026-41109) [n:127.0.0.1:33659_solr c:test_col s:shard5  x:test_col_shard5_replica2] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2205901 INFO  (qtp1780983665-41210) [n:127.0.0.1:41211_solr c:test_col s:shard2  x:test_col_shard2_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2205901 INFO  (qtp1780983665-41210) [n:127.0.0.1:41211_solr c:test_col s:shard2  x:test_col_shard2_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2205903 INFO  (qtp1000444165-41143) [n:127.0.0.1:41398_solr c:test_col s:shard4  x:test_col_shard4_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2205903 INFO  (qtp1000444165-41143) [n:127.0.0.1:41398_solr c:test_col s:shard4  x:test_col_shard4_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2205930 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2205930 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2205931 INFO  (qtp1780983665-41210) [n:127.0.0.1:41211_solr c:test_col s:shard2  x:test_col_shard2_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2205931 INFO  (qtp1780983665-41210) [n:127.0.0.1:41211_solr c:test_col s:shard2  x:test_col_shard2_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2205932 INFO  (qtp1000444165-41143) [n:127.0.0.1:41398_solr c:test_col s:shard4  x:test_col_shard4_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2205932 INFO  (qtp1000444165-41143) [n:127.0.0.1:41398_solr c:test_col s:shard4  x:test_col_shard4_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2205934 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2205934 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2205935 INFO  (qtp1780983665-41210) [n:127.0.0.1:41211_solr c:test_col s:shard2  x:test_col_shard2_replica2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=95.7900390625, floorSegmentMB=1.2060546875, forceMergeDeletesPctAllowed=13.938660722459009, segmentsPerTier=18.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.593659139203531
   [junit4]   2> 2205935 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=95.7900390625, floorSegmentMB=1.2060546875, forceMergeDeletesPctAllowed=13.938660722459009, segmentsPerTier=18.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.593659139203531
   [junit4]   2> 2205935 INFO  (qtp1780983665-41210) [n:127.0.0.1:41211_solr c:test_col s:shard2  x:test_col_shard2_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@2027c6d8[test_col_shard2_replica2] main]
   [junit4]   2> 2205936 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@36bd9dd[test_col_shard1_replica2] main]
   [junit4]   2> 2205937 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set
   [junit4]   2> 2205937 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set
   [junit4]   2> 2205938 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:shard1  x:test_col_shard1_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2205938 INFO  (qtp67249481-41118) [n:127.0.0.1:44751_solr c:test_col s:sha

[...truncated too long message...]

 (5) -> (4)
   [junit4]   2> 2371926 INFO  (zkCallback-1460-thread-8-processing-n:127.0.0.1:33659_solr) [n:127.0.0.1:33659_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 2371927 INFO  (zkCallback-1472-thread-4-processing-n:127.0.0.1:40738_solr) [n:127.0.0.1:40738_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 2371927 INFO  (jetty-closer-1430-thread-5) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1b4d091a{/solr,null,UNAVAILABLE}
   [junit4]   2> 2371927 INFO  (jetty-closer-1430-thread-10) [    ] o.a.s.c.Overseer Overseer (id=97751806434148367-127.0.0.1:33659_solr-n_0000000000) closing
   [junit4]   2> 2371927 INFO  (OverseerStateUpdate-97751806434148367-127.0.0.1:33659_solr-n_0000000000) [n:127.0.0.1:33659_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:33659_solr
   [junit4]   2> 2371961 INFO  (zkCallback-1460-thread-8-processing-n:127.0.0.1:33659_solr) [n:127.0.0.1:33659_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (1)
   [junit4]   2> 2371962 WARN  (zkCallback-1460-thread-8-processing-n:127.0.0.1:33659_solr) [n:127.0.0.1:33659_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 2371962 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b48c002a9000f, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2372674 WARN  (zkCallback-1487-thread-2-processing-n:127.0.0.1:59985_solr) [n:127.0.0.1:59985_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 2372676 INFO  (jetty-closer-1430-thread-8) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@38450000{/solr,null,UNAVAILABLE}
   [junit4]   2> 2372742 WARN  (zkCallback-1480-thread-3-processing-n:127.0.0.1:41398_solr) [n:127.0.0.1:41398_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 2372744 INFO  (jetty-closer-1430-thread-3) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@62a505a4{/solr,null,UNAVAILABLE}
   [junit4]   2> 2373424 WARN  (zkCallback-1456-thread-3-processing-n:127.0.0.1:44751_solr) [n:127.0.0.1:44751_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 2373427 INFO  (jetty-closer-1430-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6a80360c{/solr,null,UNAVAILABLE}
   [junit4]   2> 2373459 WARN  (zkCallback-1464-thread-3-processing-n:127.0.0.1:46611_solr) [n:127.0.0.1:46611_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 2373459 WARN  (zkCallback-1472-thread-4-processing-n:127.0.0.1:40738_solr) [n:127.0.0.1:40738_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 2373460 INFO  (jetty-closer-1430-thread-7) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@73ab2b56{/solr,null,UNAVAILABLE}
   [junit4]   2> 2373460 INFO  (jetty-closer-1430-thread-4) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@16cf49e1{/solr,null,UNAVAILABLE}
   [junit4]   2> 2373460 WARN  (zkCallback-1486-thread-5-processing-n:127.0.0.1:55816_solr) [n:127.0.0.1:55816_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 2373461 INFO  (jetty-closer-1430-thread-9) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@26023047{/solr,null,UNAVAILABLE}
   [junit4]   2> 2373464 INFO  (jetty-closer-1430-thread-10) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5810158{/solr,null,UNAVAILABLE}
   [junit4]   2> 2373466 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[21B1B767741BFF2]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44900 44900
   [junit4]   2> 2373565 INFO  (Thread-33801) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44900 44900
   [junit4]   2> 2373567 WARN  (Thread-33801) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 11 /solr/aliases.json
   [junit4]   2> 11 /solr/clusterprops.json
   [junit4]   2> 10 /solr/configs/org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set
   [junit4]   2> 10 /solr/security.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 11 /solr/clusterstate.json
   [junit4]   2> 10 /solr/collections/test_col/state.json
   [junit4]   2> 3 /solr/overseer_elect/election/97751806434148370-127.0.0.1:46611_solr-n_0000000005
   [junit4]   2> 2 /solr/overseer_elect/election/97751806434148379-127.0.0.1:37196_solr-n_0000000003
   [junit4]   2> 2 /solr/overseer_elect/election/97751806434148367-127.0.0.1:33659_solr-n_0000000000
   [junit4]   2> 2 /solr/overseer_elect/election/97751806434148380-127.0.0.1:41211_solr-n_0000000007
   [junit4]   2> 2 /solr/overseer_elect/election/97751806434148377-127.0.0.1:40738_solr-n_0000000002
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 165 /solr/overseer/collection-queue-work
   [junit4]   2> 16 /solr/overseer/queue
   [junit4]   2> 11 /solr/live_nodes
   [junit4]   2> 11 /solr/collections
   [junit4]   2> 4 /solr/overseer/queue-work
   [junit4]   2>
   [junit4]   2> 2373569 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[21B1B767741BFF2]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_21B1B767741BFF2-001
   [junit4]   2> Apr 07, 2017 2:12:48 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {long_stored_idx=BlockTreeOrds(blocksize=128), long_dv_idx=PostingsFormat(name=Direct), id=PostingsFormat(name=LuceneVarGapDocFreqInterval), long_dv_stored_idx=PostingsFormat(name=Direct)}, docValues:{_version_=DocValuesFormat(name=Memory), long_dv_idx=DocValuesFormat(name=Lucene70), long_dv_stored=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Lucene70), long_dv=DocValuesFormat(name=Lucene70), long_dv_stored_idx=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=511, maxMBSortInHeap=6.9818288026923945, sim=RandomSimilarity(queryNorm=false): {}, locale=ro-RO, timezone=America/Glace_Bay
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=257827336,total=531628032
   [junit4]   2> NOTE: All tests run in this JVM: [TestBM25SimilarityFactory, TestSolrCloudWithDelegationTokens, JSONWriterTest, UpdateLogTest, SimplePostToolTest, LoggingHandlerTest, TolerantUpdateProcessorTest, DateRangeFieldTest, SuggesterFSTTest, CdcrRequestHandlerTest, CursorMarkTest, TestSolr4Spatial2, PropertiesRequestHandlerTest, TestExclusionRuleCollectionAccess, TestSlowCompositeReaderWrapper, TestChildDocTransformer, SolrShardReporterTest, TermVectorComponentTest, ConvertedLegacyTest, TestSolrCloudWithHadoopAuthPlugin, FileUtilsTest, TestXIncludeConfig, ForceLeaderTest, SampleTest, RuleEngineTest, TimeZoneUtilsTest, DistributedTermsComponentTest, TestQueryUtils, TestPerFieldSimilarity, TestSegmentSorting, UpdateRequestProcessorFactoryTest, DistanceFunctionTest, TestLegacyNumericRangeQueryBuilder, SpellingQueryConverterTest, TestDynamicFieldResource, ChaosMonkeySafeLeaderTest, JavabinLoaderTest, TestCursorMarkWithoutUniqueKey, SolrInfoMBeanTest, DistributedMLTComponentTest, TestLeaderElectionZkExpiry, DocumentBuilderTest, ConnectionManagerTest, DeleteInactiveReplicaTest, SpellCheckComponentTest, TestConfigOverlay, CdcrBootstrapTest, ManagedSchemaRoundRobinCloudTest, TestInitParams, HLLUtilTest, CollectionsAPIDistributedZkTest, WordBreakSolrSpellCheckerTest, TestHashPartitioner, TestStressVersions, TestUpdate, TestAuthenticationFramework, SolrIndexSplitterTest, SuggestComponentContextFilterQueryTest, ConjunctionSolrSpellCheckerTest, HdfsBasicDistributedZkTest, DocValuesTest, TestSurroundQueryParser, HdfsDirectoryTest, TestUniqueKeyFieldResource, TestConfigSetsAPIExclusivity, TestExpandComponent, HdfsThreadLeakTest, TestStandardQParsers, TestCharFilters, TestDynamicLoading, ZkControllerTest, TestHdfsUpdateLog, TestCloudSchemaless, SpatialFilterTest, DistribCursorPagingTest, TestSha256AuthenticationProvider, BlockCacheTest, DistributedQueueTest, RequestLoggingTest, TestFieldCacheSortRandom, IgnoreCommitOptimizeUpdateProcessorFactoryTest, IndexSchemaRuntimeFieldTest, HdfsCollectionsAPIDistributedZkTest, LeaderFailoverAfterPartitionTest, SortByFunctionTest, CollectionsAPISolrJTest, SSLMigrationTest, SolrMetricsIntegrationTest, TestStressCloudBlindAtomicUpdates]
   [junit4] Completed [264/703 (1!)] on J1 in 174.40s, 5 tests, 1 error <<< FAILURES!

[...truncated 44734 lines...]




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1278 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1278/

8 tests failed.
FAILED:  org.apache.solr.DistributedIntervalFacetingTest.test

Error Message:
Error from server at http://127.0.0.1:50986//collection1: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.DistributedIntervalFacetingTest_7CCA543C436F2E33-001/index-NIOFSDirectory-016/_15_Direct_0.dvdd: Too many open files

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:50986//collection1: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.DistributedIntervalFacetingTest_7CCA543C436F2E33-001/index-NIOFSDirectory-016/_15_Direct_0.dvdd: Too many open files
        at __randomizedtesting.SeedInfo.seed([7CCA543C436F2E33:F49E6BE6ED9343CB]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
        at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:484)
        at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:463)
        at org.apache.solr.BaseDistributedSearchTestCase.commit(BaseDistributedSearchTestCase.java:549)
        at org.apache.solr.DistributedIntervalFacetingTest.testRandom(DistributedIntervalFacetingTest.java:148)
        at org.apache.solr.DistributedIntervalFacetingTest.test(DistributedIntervalFacetingTest.java:46)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsRepeatStatement.callStatement(BaseDistributedSearchTestCase.java:1011)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.TestDistributedSearch.test

Error Message:
Expected to find shardAddress in the up shard info: {error=org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request,trace=org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request  at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:421)  at org.apache.solr.handler.component.HttpShardHandlerFactory.makeLoadBalancedRequest(HttpShardHandlerFactory.java:255)  at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:166)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:745) ,time=1}

Stack Trace:
java.lang.AssertionError: Expected to find shardAddress in the up shard info: {error=org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request,trace=org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:421)
        at org.apache.solr.handler.component.HttpShardHandlerFactory.makeLoadBalancedRequest(HttpShardHandlerFactory.java:255)
        at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:166)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
,time=1}
        at __randomizedtesting.SeedInfo.seed([7CCA543C436F2E33:F49E6BE6ED9343CB]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.TestDistributedSearch.comparePartialResponses(TestDistributedSearch.java:1186)
        at org.apache.solr.TestDistributedSearch.queryPartialResults(TestDistributedSearch.java:1127)
        at org.apache.solr.TestDistributedSearch.test(TestDistributedSearch.java:987)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsRepeatStatement.callStatement(BaseDistributedSearchTestCase.java:1011)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
Expected 2 of 3 replicas to be active but only found 1; [core_node2:{"core":"c8n_1x3_lf_shard1_replica1","base_url":"http://127.0.0.1:43430/vi_vma","node_name":"127.0.0.1:43430_vi_vma","state":"active","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/31)={   "replicationFactor":"3",   "shards":{"shard1":{       "range":"80000000-7fffffff",       "state":"active",       "replicas":{         "core_node1":{           "state":"down",           "base_url":"http://127.0.0.1:40640/vi_vma",           "core":"c8n_1x3_lf_shard1_replica3",           "node_name":"127.0.0.1:40640_vi_vma"},         "core_node2":{           "core":"c8n_1x3_lf_shard1_replica1",           "base_url":"http://127.0.0.1:43430/vi_vma",           "node_name":"127.0.0.1:43430_vi_vma",           "state":"active",           "leader":"true"},         "core_node3":{           "core":"c8n_1x3_lf_shard1_replica2",           "base_url":"http://127.0.0.1:40241/vi_vma",           "node_name":"127.0.0.1:40241_vi_vma",           "state":"down"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"1",   "autoAddReplicas":"false",   "realtimeReplicas":"-1"}

Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 1; [core_node2:{"core":"c8n_1x3_lf_shard1_replica1","base_url":"http://127.0.0.1:43430/vi_vma","node_name":"127.0.0.1:43430_vi_vma","state":"active","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/31)={
  "replicationFactor":"3",
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{
        "core_node1":{
          "state":"down",
          "base_url":"http://127.0.0.1:40640/vi_vma",
          "core":"c8n_1x3_lf_shard1_replica3",
          "node_name":"127.0.0.1:40640_vi_vma"},
        "core_node2":{
          "core":"c8n_1x3_lf_shard1_replica1",
          "base_url":"http://127.0.0.1:43430/vi_vma",
          "node_name":"127.0.0.1:43430_vi_vma",
          "state":"active",
          "leader":"true"},
        "core_node3":{
          "core":"c8n_1x3_lf_shard1_replica2",
          "base_url":"http://127.0.0.1:40241/vi_vma",
          "node_name":"127.0.0.1:40241_vi_vma",
          "state":"down"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "realtimeReplicas":"-1"}
        at __randomizedtesting.SeedInfo.seed([7CCA543C436F2E33:F49E6BE6ED9343CB]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:170)
        at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:57)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv_stored

Error Message:
There are still nodes recoverying - waited for 330 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
        at __randomizedtesting.SeedInfo.seed([7CCA543C436F2E33:EEF635B8FF2813EB]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:187)
        at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.waitForRecoveriesToFinish(TestStressCloudBlindAtomicUpdates.java:459)
        at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:304)
        at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv_stored(TestStressCloudBlindAtomicUpdates.java:203)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.schema.TestPointFields.testDoublePointFieldReturn

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
        at __randomizedtesting.SeedInfo.seed([7CCA543C436F2E33:49B1140149781C9D]:0)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:882)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:849)
        at org.apache.solr.schema.TestPointFields.testPointFieldReturn(TestPointFields.java:1146)
        at org.apache.solr.schema.TestPointFields.testDoublePointFieldReturn(TestPointFields.java:257)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result/doc[1]/double[@name='number_p_d'][.='7.929467157567953']
        xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="41" start="0"><doc><str name="id">8</str><double name="number_p_d">1.5323452779726598</double></doc><doc><str name="id">9</str><double name="number_p_d">8.338396786658262</double></doc><doc><str name="id">10</str><double name="number_p_d">0.8848481289471721</double></doc><doc><str name="id">11</str><double name="number_p_d">1.788187818010275</double></doc><doc><str name="id">12</str><double name="number_p_d">9.940386508453189</double></doc><doc><str name="id">13</str><double name="number_p_d">9.230437297186704</double></doc><doc><str name="id">14</str><double name="number_p_d">0.03695443686259847</double></doc><doc><str name="id">15</str><double name="number_p_d">6.062402222909059</double></doc><doc><str name="id">0</str><double name="number_p_d">7.929467157567953</double></doc><doc><str name="id">1</str><double name="number_p_d">2.8518862278316472</double></doc><doc><str name="id">2</str><double name="number_p_d">9.232377201018501</double></doc><doc><str name="id">3</str><double name="number_p_d">5.523437785026735</double></doc><doc><str name="id">4</str><double name="number_p_d">2.5495248593851816</double></doc><doc><str name="id">5</str><double name="number_p_d">2.595282295313731</double></doc><doc><str name="id">6</str><double name="number_p_d">1.706901633525536</double></doc><doc><str name="id">7</str><double name="number_p_d">8.337614429097936</double></doc><doc><str name="id">16</str><double name="number_p_d">4.192267158889546</double></doc><doc><str name="id">17</str><double name="number_p_d">6.036500637361764</double></doc><doc><str name="id">18</str><double name="number_p_d">7.045082780424822</double></doc><doc><str name="id">19</str><double name="number_p_d">2.659311525644993</double></doc><doc><str name="id">20</str><double name="number_p_d">7.846652127170829</double></doc><doc><str name="id">21</str><double name="number_p_d">9.153879325947273</double></doc><doc><str name="id">22</str><double name="number_p_d">7.278516639091967</double></doc><doc><str name="id">23</str><double name="number_p_d">2.0359975457885437</double></doc><doc><str name="id">24</str><double name="number_p_d">6.695037886985271</double></doc><doc><str name="id">25</str><double name="number_p_d">5.457626987726848</double></doc><doc><str name="id">26</str><double name="number_p_d">6.042941653007899</double></doc><doc><str name="id">27</str><double name="number_p_d">5.409750064912645</double></doc><doc><str name="id">28</str><double name="number_p_d">4.621185908474413</double></doc><doc><str name="id">29</str><double name="number_p_d">6.096017811087675</double></doc><doc><str name="id">30</str><double name="number_p_d">0.008110482957183196</double></doc><doc><str name="id">31</str><double name="number_p_d">7.6841922418455155</double></doc><doc><str name="id">32</str><double name="number_p_d">5.213670171712261</double></doc><doc><str name="id">33</str><double name="number_p_d">4.280634027992924</double></doc><doc><str name="id">34</str><double name="number_p_d">9.339717870434551</double></doc><doc><str name="id">35</str><double name="number_p_d">6.137897506380662</double></doc><doc><str name="id">36</str><double name="number_p_d">1.4642296339962535</double></doc><doc><str name="id">37</str><double name="number_p_d">2.7908718306011293</double></doc><doc><str name="id">38</str><double name="number_p_d">4.477171206737019</double></doc><doc><str name="id">39</str><double name="number_p_d">9.996703701243861</double></doc><doc><str name="id">40</str><double name="number_p_d">5.896233962122022</double></doc></result>
</response>

        request was:q=*:*&fl=id,+number_p_d&rows=41&wt=xml
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:875)
        ... 42 more


FAILED:  org.apache.solr.schema.TestPointFields.testFloatPointFieldReturn

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
        at __randomizedtesting.SeedInfo.seed([7CCA543C436F2E33:4D559A37318D87D4]:0)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:882)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:849)
        at org.apache.solr.schema.TestPointFields.testPointFieldReturn(TestPointFields.java:1146)
        at org.apache.solr.schema.TestPointFields.testFloatPointFieldReturn(TestPointFields.java:446)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result/doc[1]/float[@name='number_p_f'][.='0.4909742']
        xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="44" start="0"><doc><str name="id">8</str><float name="number_p_f">4.1860685</float></doc><doc><str name="id">9</str><float name="number_p_f">0.59685767</float></doc><doc><str name="id">10</str><float name="number_p_f">4.506048</float></doc><doc><str name="id">11</str><float name="number_p_f">7.999423</float></doc><doc><str name="id">12</str><float name="number_p_f">6.4360957</float></doc><doc><str name="id">13</str><float name="number_p_f">0.7272601</float></doc><doc><str name="id">14</str><float name="number_p_f">7.4582596</float></doc><doc><str name="id">15</str><float name="number_p_f">6.8724484</float></doc><doc><str name="id">0</str><float name="number_p_f">0.4909742</float></doc><doc><str name="id">1</str><float name="number_p_f">7.5719204</float></doc><doc><str name="id">2</str><float name="number_p_f">0.20378053</float></doc><doc><str name="id">3</str><float name="number_p_f">0.62894464</float></doc><doc><str name="id">4</str><float name="number_p_f">0.7083124</float></doc><doc><str name="id">5</str><float name="number_p_f">4.726786</float></doc><doc><str name="id">6</str><float name="number_p_f">2.1256046</float></doc><doc><str name="id">7</str><float name="number_p_f">0.8213341</float></doc><doc><str name="id">16</str><float name="number_p_f">1.6234773</float></doc><doc><str name="id">17</str><float name="number_p_f">5.9374995</float></doc><doc><str name="id">18</str><float name="number_p_f">0.9019512</float></doc><doc><str name="id">19</str><float name="number_p_f">9.160443</float></doc><doc><str name="id">20</str><float name="number_p_f">1.2007517</float></doc><doc><str name="id">21</str><float name="number_p_f">6.113803</float></doc><doc><str name="id">22</str><float name="number_p_f">5.2494755</float></doc><doc><str name="id">23</str><float name="number_p_f">8.203841</float></doc><doc><str name="id">24</str><float name="number_p_f">7.0899634</float></doc><doc><str name="id">25</str><float name="number_p_f">4.759904</float></doc><doc><str name="id">26</str><float name="number_p_f">5.7973614</float></doc><doc><str name="id">27</str><float name="number_p_f">9.73888</float></doc><doc><str name="id">28</str><float name="number_p_f">0.75692415</float></doc><doc><str name="id">29</str><float name="number_p_f">9.986328</float></doc><doc><str name="id">30</str><float name="number_p_f">3.5090904</float></doc><doc><str name="id">31</str><float name="number_p_f">4.8126516</float></doc><doc><str name="id">32</str><float name="number_p_f">5.249214</float></doc><doc><str name="id">33</str><float name="number_p_f">1.2050235</float></doc><doc><str name="id">34</str><float name="number_p_f">8.183576</float></doc><doc><str name="id">35</str><float name="number_p_f">2.9626477</float></doc><doc><str name="id">36</str><float name="number_p_f">0.6687838</float></doc><doc><str name="id">37</str><float name="number_p_f">9.69768</float></doc><doc><str name="id">38</str><float name="number_p_f">9.276188</float></doc><doc><str name="id">39</str><float name="number_p_f">8.4332075</float></doc><doc><str name="id">40</str><float name="number_p_f">7.276645</float></doc><doc><str name="id">41</str><float name="number_p_f">1.8072826</float></doc><doc><str name="id">42</str><float name="number_p_f">0.50658286</float></doc><doc><str name="id">43</str><float name="number_p_f">4.957859</float></doc></result>
</response>

        request was:q=*:*&fl=id,+number_p_f&rows=44&wt=xml
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:875)
        ... 42 more


FAILED:  org.apache.solr.schema.TestUseDocValuesAsStored.testMultipleSearchResults

Error Message:
mismatch: 'myid1'!='myid' @ response/docs/[0]/id

Stack Trace:
java.lang.RuntimeException: mismatch: 'myid1'!='myid' @ response/docs/[0]/id
        at __randomizedtesting.SeedInfo.seed([7CCA543C436F2E33:4EE053A2BB910AEA]:0)
        at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:970)
        at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:917)
        at org.apache.solr.schema.TestUseDocValuesAsStored.testMultipleSearchResults(TestUseDocValuesAsStored.java:243)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.client.solrj.TestSolrJErrorHandling.testWithXml

Error Message:
expected:<704> but was:<694>

Stack Trace:
java.lang.AssertionError: expected:<704> but was:<694>
        at __randomizedtesting.SeedInfo.seed([F31B74DA5F52583F:58F9CD2CDD544152]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.client.solrj.TestSolrJErrorHandling.doThreads(TestSolrJErrorHandling.java:185)
        at org.apache.solr.client.solrj.TestSolrJErrorHandling.doIt(TestSolrJErrorHandling.java:200)
        at org.apache.solr.client.solrj.TestSolrJErrorHandling.testWithXml(TestSolrJErrorHandling.java:110)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11697 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_7CCA543C436F2E33-001/init-core-data-001
   [junit4]   2> 70814 WARN  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[7CCA543C436F2E33]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 70815 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[7CCA543C436F2E33]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields
   [junit4]   2> 70827 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[7CCA543C436F2E33]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=SSL overhead seems to cause OutOfMemory when stress testing)
   [junit4]   2> 70829 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[7CCA543C436F2E33]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 10 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_7CCA543C436F2E33-001/tempDir-001
   [junit4]   2> 70829 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[7CCA543C436F2E33]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 70862 INFO  (Thread-20) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 70862 INFO  (Thread-20) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 70962 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[7CCA543C436F2E33]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:53836
   [junit4]   2> 71072 INFO  (jetty-launcher-16-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 71080 INFO  (jetty-launcher-16-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 71096 INFO  (jetty-launcher-16-thread-4) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 71096 INFO  (jetty-launcher-16-thread-3) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 71108 INFO  (jetty-launcher-16-thread-5) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 71127 INFO  (jetty-launcher-16-thread-6) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 71133 INFO  (jetty-launcher-16-thread-7) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 71152 INFO  (jetty-launcher-16-thread-8) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 71168 INFO  (jetty-launcher-16-thread-9) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 71416 INFO  (jetty-launcher-16-thread-10) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 71255 INFO  (jetty-launcher-16-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6dad79c0{/solr,null,AVAILABLE}
   [junit4]   2> 71859 INFO  (jetty-launcher-16-thread-7) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7267b31d{/solr,null,AVAILABLE}
   [junit4]   2> 71890 INFO  (jetty-launcher-16-thread-6) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@387df14b{/solr,null,AVAILABLE}
   [junit4]   2> 71407 INFO  (jetty-launcher-16-thread-5) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@378f14ac{/solr,null,AVAILABLE}
   [junit4]   2> 71655 INFO  (jetty-launcher-16-thread-10) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@16691a67{/solr,null,AVAILABLE}
   [junit4]   2> 71779 INFO  (jetty-launcher-16-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@13f583ef{/solr,null,AVAILABLE}
   [junit4]   2> 71991 INFO  (jetty-launcher-16-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@359cd97d{/solr,null,AVAILABLE}
   [junit4]   2> 72010 INFO  (jetty-launcher-16-thread-7) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1be6f2e9{HTTP/1.1,[http/1.1]}{127.0.0.1:44301}
   [junit4]   2> 72055 INFO  (jetty-launcher-16-thread-7) [    ] o.e.j.s.Server Started @79802ms
   [junit4]   2> 72055 INFO  (jetty-launcher-16-thread-7) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44301}
   [junit4]   2> 72055 ERROR (jetty-launcher-16-thread-7) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 72055 INFO  (jetty-launcher-16-thread-7) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 72056 INFO  (jetty-launcher-16-thread-7) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 72056 INFO  (jetty-launcher-16-thread-7) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 72056 INFO  (jetty-launcher-16-thread-7) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-09T08:58:01.648Z
   [junit4]   2> 72010 INFO  (jetty-launcher-16-thread-10) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2f9a6e58{HTTP/1.1,[http/1.1]}{127.0.0.1:42334}
   [junit4]   2> 72056 INFO  (jetty-launcher-16-thread-10) [    ] o.e.j.s.Server Started @79803ms
   [junit4]   2> 72013 INFO  (jetty-launcher-16-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@12178aa6{HTTP/1.1,[http/1.1]}{127.0.0.1:47124}
   [junit4]   2> 72013 INFO  (jetty-launcher-16-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5b66cdd4{HTTP/1.1,[http/1.1]}{127.0.0.1:51337}
   [junit4]   2> 72057 INFO  (jetty-launcher-16-thread-1) [    ] o.e.j.s.Server Started @79804ms
   [junit4]   2> 72057 INFO  (jetty-launcher-16-thread-2) [    ] o.e.j.s.Server Started @79804ms
   [junit4]   2> 72013 INFO  (jetty-launcher-16-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5b7195ff{HTTP/1.1,[http/1.1]}{127.0.0.1:46998}
   [junit4]   2> 72057 INFO  (jetty-launcher-16-thread-4) [    ] o.e.j.s.Server Started @79804ms
   [junit4]   2> 72014 INFO  (jetty-launcher-16-thread-6) [    ] o.e.j.s.AbstractConnector Started ServerConnector@311bbf16{HTTP/1.1,[http/1.1]}{127.0.0.1:41191}
   [junit4]   2> 72057 INFO  (jetty-launcher-16-thread-6) [    ] o.e.j.s.Server Started @79804ms
   [junit4]   2> 72047 INFO  (jetty-launcher-16-thread-5) [    ] o.e.j.s.AbstractConnector Started ServerConnector@139c1bbe{HTTP/1.1,[http/1.1]}{127.0.0.1:35630}
   [junit4]   2> 72057 INFO  (jetty-launcher-16-thread-5) [    ] o.e.j.s.Server Started @79804ms
   [junit4]   2> 72097 INFO  (jetty-launcher-16-thread-8) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@30f25b87{/solr,null,AVAILABLE}
   [junit4]   2> 72097 INFO  (jetty-launcher-16-thread-8) [    ] o.e.j.s.AbstractConnector Started ServerConnector@49d9ab52{HTTP/1.1,[http/1.1]}{127.0.0.1:47865}
   [junit4]   2> 72098 INFO  (jetty-launcher-16-thread-8) [    ] o.e.j.s.Server Started @79845ms
   [junit4]   2> 72123 INFO  (jetty-launcher-16-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4b2d33d9{/solr,null,AVAILABLE}
   [junit4]   2> 72123 INFO  (jetty-launcher-16-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7ce7f639{HTTP/1.1,[http/1.1]}{127.0.0.1:36724}
   [junit4]   2> 72123 INFO  (jetty-launcher-16-thread-3) [    ] o.e.j.s.Server Started @79870ms
   [junit4]   2> 72056 INFO  (jetty-launcher-16-thread-10) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42334}
   [junit4]   2> 72057 INFO  (jetty-launcher-16-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=47124}
   [junit4]   2> 72057 INFO  (jetty-launcher-16-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51337}
   [junit4]   2> 72057 INFO  (jetty-launcher-16-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46998}
   [junit4]   2> 72057 INFO  (jetty-launcher-16-thread-6) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41191}
   [junit4]   2> 72057 INFO  (jetty-launcher-16-thread-5) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35630}
   [junit4]   2> 72098 INFO  (jetty-launcher-16-thread-8) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=47865}
   [junit4]   2> 72123 INFO  (jetty-launcher-16-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36724}
   [junit4]   2> 72203 INFO  (jetty-launcher-16-thread-9) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4be24359{/solr,null,AVAILABLE}
   [junit4]   2> 72204 INFO  (jetty-launcher-16-thread-9) [    ] o.e.j.s.AbstractConnector Started ServerConnector@460d637c{HTTP/1.1,[http/1.1]}{127.0.0.1:55794}
   [junit4]   2> 72204 INFO  (jetty-launcher-16-thread-9) [    ] o.e.j.s.Server Started @79951ms
   [junit4]   2> 72204 INFO  (jetty-launcher-16-thread-9) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55794}
   [junit4]   2> 72131 ERROR (jetty-launcher-16-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 72132 ERROR (jetty-launcher-16-thread-5) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 72132 ERROR (jetty-launcher-16-thread-8) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 72147 ERROR (jetty-launcher-16-thread-6) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 72147 ERROR (jetty-launcher-16-thread-4) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 72147 ERROR (jetty-launcher-16-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 72147 ERROR (jetty-launcher-16-thread-3) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 72147 ERROR (jetty-launcher-16-thread-10) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 72204 ERROR (jetty-launcher-16-thread-9) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 72211 INFO  (jetty-launcher-16-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 72228 INFO  (jetty-launcher-16-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 72228 INFO  (jetty-launcher-16-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 72228 INFO  (jetty-launcher-16-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-09T08:58:01.820Z
   [junit4]   2> 72211 INFO  (jetty-launcher-16-thread-5) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 72229 INFO  (jetty-launcher-16-thread-5) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 72229 INFO  (jetty-launcher-16-thread-5) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 72229 INFO  (jetty-launcher-16-thread-5) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-09T08:58:01.821Z
   [junit4]   2> 72211 INFO  (jetty-launcher-16-thread-8) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 72231 INFO  (jetty-launcher-16-thread-8) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 72231 INFO  (jetty-launcher-16-thread-8) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 72231 INFO  (jetty-launcher-16-thread-8) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-09T08:58:01.823Z
   [junit4]   2> 72212 INFO  (jetty-launcher-16-thread-6) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 72235 INFO  (jetty-launcher-16-thread-6) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 72235 INFO  (jetty-launcher-16-thread-6) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 72235 INFO  (jetty-launcher-16-thread-6) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-09T08:58:01.827Z
   [junit4]   2> 72212 INFO  (jetty-launcher-16-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 72235 INFO  (jetty-launcher-16-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 72235 INFO  (jetty-launcher-16-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 72236 INFO  (jetty-launcher-16-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-09T08:58:01.828Z
   [junit4]   2> 72213 INFO  (jetty-launcher-16-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 72236 INFO  (jetty-launcher-16-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 72236 INFO  (jetty-launcher-16-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 72236 INFO  (jetty-launcher-16-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-09T08:58:01.828Z
   [junit4]   2> 72213 INFO  (jetty-launcher-16-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 72237 INFO  (jetty-launcher-16-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 72237 INFO  (jetty-launcher-16-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 72237 INFO  (jetty-launcher-16-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-09T08:58:01.829Z
   [junit4]   2> 72213 INFO  (jetty-launcher-16-thread-10) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 72237 INFO  (jetty-launcher-16-thread-10) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 72237 INFO  (jetty-launcher-16-thread-10) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 72238 INFO  (jetty-launcher-16-thread-10) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-09T08:58:01.829Z
   [junit4]   2> 72214 INFO  (jetty-launcher-16-thread-9) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 72238 INFO  (jetty-launcher-16-thread-9) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 72238 INFO  (jetty-launcher-16-thread-9) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 72238 INFO  (jetty-launcher-16-thread-9) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-09T08:58:01.830Z
   [junit4]   2> 72353 INFO  (jetty-launcher-16-thread-7) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 72460 INFO  (jetty-launcher-16-thread-5) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 72492 INFO  (jetty-launcher-16-thread-6) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 72539 INFO  (jetty-launcher-16-thread-6) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53836/solr
   [junit4]   2> 72562 INFO  (jetty-launcher-16-thread-9) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 72579 INFO  (jetty-launcher-16-thread-7) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53836/solr
   [junit4]   2> 72580 INFO  (jetty-launcher-16-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 72587 INFO  (jetty-launcher-16-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 72581 INFO  (jetty-launcher-16-thread-10) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 72581 INFO  (jetty-launcher-16-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 72581 INFO  (jetty-launcher-16-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 72622 INFO  (jetty-launcher-16-thread-5) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53836/solr
   [junit4]   2> 72623 INFO  (jetty-launcher-16-thread-8) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 72761 INFO  (jetty-launcher-16-thread-9) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53836/solr
   [junit4]   2> 72762 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b51ef2b9b0003, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 72844 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b51ef2b9b0005, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 72844 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b51ef2b9b000b, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 72845 INFO  (jetty-launcher-16-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53836/solr
   [junit4]   2> 72848 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b51ef2b9b000a, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 72885 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b51ef2b9b000c, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 72949 INFO  (jetty-launcher-16-thread-8) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53836/solr
   [junit4]   2> 72973 INFO  (jetty-launcher-16-thread-10) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53836/solr
   [junit4]   2> 73043 INFO  (jetty-launcher-16-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53836/solr
   [junit4]   2> 73132 INFO  (jetty-launcher-16-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53836/solr
   [junit4]   2> 73195 INFO  (jetty-launcher-16-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53836/solr
   [junit4]   2> 73248 INFO  (jetty-launcher-16-thread-6) [n:127.0.0.1:41191_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 73352 INFO  (jetty-launcher-16-thread-6) [n:127.0.0.1:41191_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41191_solr
   [junit4]   2> 73353 INFO  (jetty-launcher-16-thread-6) [n:127.0.0.1:41191_solr    ] o.a.s.c.Overseer Overseer (id=97761904589209615-127.0.0.1:41191_solr-n_0000000000) starting
   [junit4]   2> 73499 INFO  (jetty-launcher-16-thread-6) [n:127.0.0.1:41191_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41191_solr
   [junit4]   2> 73501 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b51ef2b9b0013, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 73503 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b51ef2b9b0010, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 73539 INFO  (jetty-launcher-16-thread-7) [n:127.0.0.1:44301_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 73547 INFO  (jetty-launcher-16-thread-7) [n:127.0.0.1:44301_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 73620 INFO  (OverseerStateUpdate-97761904589209615-127.0.0.1:41191_solr-n_0000000000) [n:127.0.0.1:41191_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 73621 INFO  (jetty-launcher-16-thread-7) [n:127.0.0.1:44301_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44301_solr
   [junit4]   2> 73632 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b51ef2b9b0016, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 73635 INFO  (zkCallback-43-thread-1-processing-n:127.0.0.1:41191_solr) [n:127.0.0.1:41191_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 73749 INFO  (jetty-launcher-16-thread-9) [n:127.0.0.1:55794_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 73764 INFO  (jetty-launcher-16-thread-9) [n:127.0.0.1:55794_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 73778 INFO  (jetty-launcher-16-thread-9) [n:127.0.0.1:55794_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55794_solr
   [junit4]   2> 73779 INFO  (jetty-launcher-16-thread-8) [n:127.0.0.1:47865_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 73784 INFO  (jetty-launcher-16-thread-5) [n:127.0.0.1:35630_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 73800 INFO  (jetty-launcher-16-thread-8) [n:127.0.0.1:47865_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 73802 INFO  (jetty-launcher-16-thread-5) [n:127.0.0.1:35630_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 73896 INFO  (zkCallback-57-thread-1-processing-n:127.0.0.1:44301_solr) [n:127.0.0.1:44301_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 73901 INFO  (jetty-launcher-16-thread-8) [n:127.0.0.1:47865_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47865_solr
   [junit4]   2> 73902 INFO  (jetty-launcher-16-thread-5) [n:127.0.0.1:35630_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35630_solr
   [junit4]   2> 73912 INFO  (jetty-launcher-16-thread-10) [n:127.0.0.1:42334_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
   [junit4]   2> 73915 INFO  (jetty-launcher-16-thread-10) [n:127.0.0.1:42334_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 73918 INFO  (jetty-launcher-16-thread-10) [n:127.0.0.1:42334_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42334_solr
   [junit4]   2> 74096 INFO  (zkCallback-65-thread-1-processing-n:127.0.0.1:47865_solr) [n:127.0.0.1:47865_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (6)
   [junit4]   2> 74096 INFO  (zkCallback-63-thread-1-processing-n:127.0.0.1:55794_solr) [n:127.0.0.1:55794_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (6)
   [junit4]   2> 74097 INFO  (zkCallback-57-thread-2-processing-n:127.0.0.1:44301_solr) [n:127.0.0.1:44301_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (6)
   [junit4]   2> 74097 INFO  (zkCallback-43-thread-1-processing-n:127.0.0.1:41191_solr) [n:127.0.0.1:41191_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (6)
   [junit4]   2> 74247 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:35630_solr) [n:127.0.0.1:35630_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (6)
   [junit4]   2> 74315 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:42334_solr) [n:127.0.0.1:42334_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
   [junit4]   2> 74420 INFO  (jetty-launcher-16-thread-2) [n:127.0.0.1:51337_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (6)
   [junit4]   2> 74423 INFO  (jetty-launcher-16-thread-4) [n:127.0.0.1:46998_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (6)
   [junit4]   2> 74424 INFO  (jetty-launcher-16-thread-1) [n:127.0.0.1:47124_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (6)
   [junit4]   2> 74427 INFO  (jetty-launcher-16-thread-3) [n:127.0.0.1:36724_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (6)
   [junit4]   2> 74429 INFO  (jetty-launcher-16-thread-2) [n:127.0.0.1:51337_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 74432 INFO  (jetty-launcher-16-thread-4) [n:127.0.0.1:46998_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 74433 INFO  (jetty-launcher-16-thread-1) [n:127.0.0.1:47124_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 74436 INFO  (jetty-launcher-16-thread-3) [n:127.0.0.1:36724_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 74437 INFO  (jetty-launcher-16-thread-2) [n:127.0.0.1:51337_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51337_solr
   [junit4]   2> 74528 INFO  (jetty-launcher-16-thread-4) [n:127.0.0.1:46998_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46998_solr
   [junit4]   2> 74530 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:42334_solr) [n:127.0.0.1:42334_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
   [junit4]   2> 74530 INFO  (zkCallback-65-thread-1-processing-n:127.0.0.1:47865_solr) [n:127.0.0.1:47865_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
   [junit4]   2> 74531 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:35630_solr) [n:127.0.0.1:35630_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
   [junit4]   2> 74531 INFO  (zkCallback-57-thread-2-processing-n:127.0.0.1:44301_solr) [n:127.0.0.1:44301_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
   [junit4]   2> 74531 INFO  (zkCallback-63-thread-1-processing-n:127.0.0.1:55794_solr) [n:127.0.0.1:55794_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
   [junit4]   2> 74540 INFO  (zkCallback-43-thread-3-processing-n:127.0.0.1:41191_solr) [n:127.0.0.1:41191_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
   [junit4]   2> 74540 INFO  (jetty-launcher-16-thread-1) [n:127.0.0.1:47124_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47124_solr
   [junit4]   2> 74615 INFO  (zkCallback-66-thread-1-processing-n:127.0.0.1:51337_solr) [n:127.0.0.1:51337_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
   [junit4]   2> 74616 INFO  (zkCallback-69-thread-1-processing-n:127.0.0.1:47124_solr) [n:127.0.0.1:47124_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
   [junit4]   2> 74616 INFO  (zkCallback-70-thread-1-processing-n:127.0.0.1:36724_solr) [n:127.0.0.1:36724_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
   [junit4]   2> 74616 INFO  (zkCallback-67-thread-1-processing-n:127.0.0.1:46998_solr) [n:127.0.0.1:46998_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
   [junit4]   2> 74620 INFO  (zkCallback-57-thread-2-processing-n:127.0.0.1:44301_solr) [n:127.0.0.1:44301_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (8)
   [junit4]   2> 74622 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:35630_solr) [n:127.0.0.1:35630_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (8)
   [junit4]   2> 74622 INFO  (zkCallback-63-thread-1-processing-n:127.0.0.1:55794_solr) [n:127.0.0.1:55794_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (8)
   [junit4]   2> 74622 INFO  (zkCallback-65-thread-1-processing-n:127.0.0.1:47865_solr) [n:127.0.0.1:47865_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (8)
   [junit4]   2> 74622 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:42334_solr) [n:127.0.0.1:42334_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (8)
   [junit4]   2> 74624 INFO  (zkCallback-43-thread-3-processing-n:127.0.0.1:41191_solr) [n:127.0.0.1:41191_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (8)
   [junit4]   2> 74627 INFO  (zkCallback-57-thread-2-processing-n:127.0.0.1:44301_solr) [n:127.0.0.1:44301_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (9)
   [junit4]   2> 74628 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:42334_solr) [n:127.0.0.1:42334_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (9)
   [junit4]   2> 74628 INFO  (jetty-launcher-16-thread-3) [n:127.0.0.1:36724_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36724_solr
   [junit4]   2> 74628 INFO  (zkCallback-65-thread-1-processing-n:127.0.0.1:47865_solr) [n:127.0.0.1:47865_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (9)
   [junit4]   2> 74628 INFO  (zkCallback-63-thread-1-processing-n:127.0.0.1:55794_solr) [n:127.0.0.1:55794_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (9)
   [junit4]   2> 74629 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:35630_solr) [n:127.0.0.1:35630_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (9)
   [junit4]   2> 74635 INFO  (zkCallback-43-thread-3-processing-n:127.0.0.1:41191_solr) [n:127.0.0.1:41191_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (9)
   [junit4]   2> 74771 INFO  (zkCallback-69-thread-2-processing-n:127.0.0.1:47124_solr) [n:127.0.0.1:47124_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (10)
   [junit4]   2> 74772 INFO  (zkCallback-64-thread-1-processing-n:127.0.0.1:42334_solr) [n:127.0.0.1:42334_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
   [junit4]   2> 74773 INFO  (zkCallback-62-thread-1-processing-n:127.0.0.1:35630_solr) [n:127.0.0.1:35630_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
   [junit4]   2> 74773 INFO  (zkCallback-57-thread-2-processing-n:127.0.0.1:44301_solr) [n:127.0.0.1:44301_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
   [junit4]   2> 74773 INFO  (zkCallback-63-thread-1-processing-n:127.0.0.1:55794_solr) [n:127.0.0.1:55794_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
   [junit4]   2> 74773 INFO  (zkCallback-65-thread-1-processing-n:127.0.0.1:47865_solr) [n:127.0.0.1:47865_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
   [junit4]   2> 74774 INFO  (zkCallback-43-thread-3-processing-n:127.0.0.1:41191_solr) [n:127.0.0.1:41191_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
   [junit4]   2> 74775 INFO  (zkCallback-66-thread-2-processing-n:127.0.0.1:51337_solr) [n:127.0.0.1:51337_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (10)
   [junit4]   2> 74779 INFO  (zkCallback-70-thread-2-processing-n:127.0.0.1:36724_solr) [n:127.0.0.1:36724_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (10)
   [junit4]   2> 74905 INFO  (zkCallback-67-thread-2-processing-n:127.0.0.1:46998_solr) [n:127.0.0.1:46998_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (10)
   [junit4]   2> 74995 INFO  (jetty-launcher-16-thread-8) [n:127.0.0.1:47865_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_7CCA543C436F2E33-001/tempDir-001/node8/.
   [junit4]   2> 75011 INFO  (jetty-launcher-16-thread-7) [n:127.0.0.1:44301_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_7CCA543C436F2E33-001/tempDir-001/node7/.
   [junit4]   2> 75578 INFO  (jetty-launcher-16-thread-6) [n:127.0.0.1:41191_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_7CCA543C436F2E33-001/tempDir-001/node6/.
   [junit4]   2> 75593 INFO  (jetty-launcher-16-thread-9) [n:127.0.0.1:55794_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_7CCA543C436F2E33-001/tempDir-001/node9/.
   [junit4]   2> 75602 INFO  (jetty-launcher-16-thread-5) [n:127.0.0.1:35630_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_7CCA543C436F2E33-001/tempDir-001/node5/.
   [junit4]   2> 75659 INFO  (jetty-launcher-16-thread-10) [n:127.0.0.1:42334_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_7CCA543C436F2E33-001/tempDir-001/node10/.
   [junit4]   2> 75665 INFO  (jetty-launcher-16-thread-3) [n:127.0.0.1:36724_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_7CCA543C436F2E33-001/tempDir-001/node3/.
   [junit4]   2> 75762 INFO  (jetty-launcher-16-thread-1) [n:127.0.0.1:47124_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_7CCA543C436F2E33-001/tempDir-001/node1/.
   [junit4]   2> 75767 INFO  (jetty-launcher-16-thread-4) [n:127.0.0.1:46998_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_7CCA543C436F2E33-001/tempDir-001/node4/.
   [junit4]   2> 75767 INFO  (jetty-launcher-16-thread-2) [n:127.0.0.1:51337_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_7CCA543C436F2E33-001/tempDir-001/node2/.
   [junit4]   2> 76035 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[7CCA543C436F2E33]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (10)
   [junit4]   2> 76036 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[7CCA543C436F2E33]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53836/solr ready
   [junit4]   2> 77269 INFO  (qtp1025092489-175) [n:127.0.0.1:47865_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&name=test_col&action=CREATE&numShards=5&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 77280 INFO  (OverseerThreadFactory-71-thread-1-processing-n:127.0.0.1:41191_solr) [n:127.0.0.1:41191_solr    ] o.a.s.c.CreateCollectionCmd Create collection test_col
   [junit4]   2> 77637 INFO  (qtp1025092489-123) [n:127.0.0.1:47865_solr    ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&name=test_col_shard1_replica1&action=CREATE&numShards=5&collection=test_col&shard=shard1&wt=javabin&version=2
   [junit4]   2> 77638 INFO  (qtp1025092489-123) [n:127.0.0.1:47865_solr    ] o.a.s.c.TransientSolrCoreCach

[...truncated too long message...]

LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]:   index before flush _1(7.0.0):C694/693:delGen=1 _2(7.0.0):C10/10:delGen=1
   [junit4]   2> 204215 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-349-thread-1]: startFullFlush
   [junit4]   2> 204215 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-349-thread-1]: coreCloseExecutor-349-thread-1 finishFullFlush success=true
   [junit4]   2> 204215 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: apply all deletes during flush
   [junit4]   2> 204215 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: now apply all deletes for all segments maxDoc=704
   [junit4]   2> 204215 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-349-thread-1]: applyDeletes: open segment readers took 0 msec
   [junit4]   2> 204215 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-349-thread-1]: applyDeletes: no segments; skipping
   [junit4]   2> 204215 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-349-thread-1]: prune sis=segments_4: _1(7.0.0):C694/693:delGen=1 _2(7.0.0):C10/10:delGen=1 minGen=15 packetCount=0
   [junit4]   2> 204215 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-349-thread-1]: updateMergeThreads ioThrottle=true targetMBPerSec=10240.0 MB/sec
   [junit4]   2> 204215 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-349-thread-1]: now merge
   [junit4]   2> 204215 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-349-thread-1]:   index: _1(7.0.0):C694/693:delGen=1 _2(7.0.0):C10/10:delGen=1
   [junit4]   2> 204215 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-349-thread-1]:   no more merges pending; now return
   [junit4]   2> 204215 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: waitForMerges
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: waitForMerges done
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: commit: start
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: commit: enter lock
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: commit: now prepare
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: prepareCommit: flush
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]:   index before flush _1(7.0.0):C694/693:delGen=1 _2(7.0.0):C10/10:delGen=1
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-349-thread-1]: startFullFlush
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: apply all deletes during flush
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: now apply all deletes for all segments maxDoc=704
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-349-thread-1]: applyDeletes: open segment readers took 0 msec
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-349-thread-1]: applyDeletes: no segments; skipping
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-349-thread-1]: prune sis=segments_4: _1(7.0.0):C694/693:delGen=1 _2(7.0.0):C10/10:delGen=1 minGen=15 packetCount=0
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-349-thread-1]: coreCloseExecutor-349-thread-1 finishFullFlush success=true
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: startCommit(): start
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]:   skip startCommit(): no changes pending
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-349-thread-1]: delete []
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: commit: pendingCommit == null; skip
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: commit: took 0.4 msec
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: commit: done
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: rollback
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: all running merges have aborted
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: rollback: done finish merges
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-349-thread-1]: abort
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-349-thread-1]: done abort success=true
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-349-thread-1]: rollback: infos=_1(7.0.0):C694/693:delGen=1 _2(7.0.0):C10/10:delGen=1
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-349-thread-1]: now checkpoint "_1(7.0.0):C694/693:delGen=1 _2(7.0.0):C10/10:delGen=1" [2 segments ; isCommit = false]
   [junit4]   2> 204216 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-349-thread-1]: delete []
   [junit4]   2> 204217 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-349-thread-1]: 0 msec to checkpoint
   [junit4]   2> 204227 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-349-thread-1]: delete []
   [junit4]   2> 204227 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-349-thread-1]: delete []
   [junit4]   2> 204301 INFO  (coreCloseExecutor-349-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1555188663
   [junit4]   2> 204317 INFO  (SUITE-TestSolrJErrorHandling-seed#[F31B74DA5F52583F]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@343411fc{/solr,null,UNAVAILABLE}
   [junit4]   2> 204322 INFO  (SUITE-TestSolrJErrorHandling-seed#[F31B74DA5F52583F]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestSolrJErrorHandling_F31B74DA5F52583F-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1975, maxMBSortInHeap=7.481226886027924, sim=RandomSimilarity(queryNorm=true): {}, locale=th-TH, timezone=Pacific/Kwajalein
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=244779816,total=351272960
   [junit4]   2> NOTE: All tests run in this JVM: [JdbcDriverTest, ShardParamsTest, StreamExpressionToExplanationTest, TestClusteringResponse, CollectionAdminRequestRequiredParamsTest, TestToleratedUpdateError, TestEmbeddedSolrServer, CoalesceEvaluatorTest, TestDelegationTokenRequest, TestLBHttpSolrClient, CloudSolrClientBuilderTest, TestZkConfigManager, TermsResponseTest, AnlysisResponseBaseTest, NaturalLogEvaluatorTest, LBHttpSolrClientBuilderTest, ExclusiveOrEvaluatorTest, FieldAnalysisResponseTest, GreaterThanEqualToEvaluatorTest, LessThanEvaluatorTest, CubedRootEvaluatorTest, SolrExampleStreamingTest, SolrExampleStreamingBinaryTest, SolrExampleBinaryTest, SolrExampleEmbeddedTest, LargeVolumeEmbeddedTest, LargeVolumeJettyTest, SolrParamTest, MergeIndexesEmbeddedTest, TestNamedListCodec, TestJavaBinCodec, ContentStreamTest, ModifiableSolrParamsTest, TestXMLEscaping, DocumentAnalysisResponseTest, SolrDocumentTest, TestSolrJErrorHandling]
   [junit4] Completed [81/126 (1!)] on J0 in 38.46s, 4 tests, 1 failure <<< FAILURES!

[...truncated 42639 lines...]




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1279 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1279/

4 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI

Error Message:
expected:<3> but was:<1>

Stack Trace:
java.lang.AssertionError: expected:<3> but was:<1>
        at __randomizedtesting.SeedInfo.seed([3BBD3F23147BAA1F:73C84B971248858A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:523)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test.test

Error Message:
.response.numFound:36!=67

Stack Trace:
junit.framework.AssertionFailedError: .response.numFound:36!=67
        at __randomizedtesting.SeedInfo.seed([3BBD3F23147BAA1F:B3E900F9BA87C7E7]:0)
        at junit.framework.Assert.fail(Assert.java:50)
        at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:913)
        at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:932)
        at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:607)
        at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:587)
        at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:566)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkQueries(AbstractFullDistribZkTestBase.java:883)
        at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:98)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest

Error Message:
ObjectTracker found 3 object(s) that were not released!!! [SolrCore, NRTCachingDirectory, NRTCachingDirectory] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1022)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:849)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:964)  at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:586)  at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:745)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)  at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:353)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:715)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:931)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:849)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:964)  at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:586)  at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:745)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)  at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:92)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:747)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:931)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:849)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:964)  at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:586)  at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:745)  

Stack Trace:
java.lang.AssertionError: ObjectTracker found 3 object(s) that were not released!!! [SolrCore, NRTCachingDirectory, NRTCachingDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
        at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1022)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:849)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:964)
        at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:586)
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
        at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
        at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:353)
        at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:715)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:931)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:849)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:964)
        at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:586)
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
        at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
        at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:92)
        at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:747)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:931)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:849)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:964)
        at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:586)
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)


        at __randomizedtesting.SeedInfo.seed([3BBD3F23147BAA1F]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:301)
        at sun.reflect.GeneratedMethodAccessor112.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI

Error Message:
expected:<3> but was:<2>

Stack Trace:
java.lang.AssertionError: expected:<3> but was:<2>
        at __randomizedtesting.SeedInfo.seed([3BBD3F23147BAA1F:73C84B971248858A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:523)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12553 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_3BBD3F23147BAA1F-001/init-core-data-001
   [junit4]   2> 2860798 WARN  (SUITE-CollectionsAPIDistributedZkTest-seed#[3BBD3F23147BAA1F]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=36 numCloses=36
   [junit4]   2> 2860799 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[3BBD3F23147BAA1F]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields
   [junit4]   2> 2860800 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[3BBD3F23147BAA1F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 2860802 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[3BBD3F23147BAA1F]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_3BBD3F23147BAA1F-001/tempDir-001
   [junit4]   2> 2860802 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[3BBD3F23147BAA1F]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2860802 INFO  (Thread-8288) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2860802 INFO  (Thread-8288) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2860903 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[3BBD3F23147BAA1F]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:42919
   [junit4]   2> 2860928 INFO  (jetty-launcher-2846-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2860930 INFO  (jetty-launcher-2846-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2860931 INFO  (jetty-launcher-2846-thread-3) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2860933 INFO  (jetty-launcher-2846-thread-4) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2860934 INFO  (jetty-launcher-2846-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@704fb775{/solr,null,AVAILABLE}
   [junit4]   2> 2860935 INFO  (jetty-launcher-2846-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@527ed307{SSL,[ssl, http/1.1]}{127.0.0.1:38210}
   [junit4]   2> 2860935 INFO  (jetty-launcher-2846-thread-2) [    ] o.e.j.s.Server Started @2867449ms
   [junit4]   2> 2860935 INFO  (jetty-launcher-2846-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38210}
   [junit4]   2> 2860935 ERROR (jetty-launcher-2846-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2860935 INFO  (jetty-launcher-2846-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2860935 INFO  (jetty-launcher-2846-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2860935 INFO  (jetty-launcher-2846-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2860935 INFO  (jetty-launcher-2846-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-11T04:52:23.370Z
   [junit4]   2> 2861053 INFO  (jetty-launcher-2846-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3273665{/solr,null,AVAILABLE}
   [junit4]   2> 2861053 INFO  (jetty-launcher-2846-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2bbeef0d{SSL,[ssl, http/1.1]}{127.0.0.1:45966}
   [junit4]   2> 2861053 INFO  (jetty-launcher-2846-thread-4) [    ] o.e.j.s.Server Started @2867568ms
   [junit4]   2> 2861053 INFO  (jetty-launcher-2846-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45966}
   [junit4]   2> 2861054 ERROR (jetty-launcher-2846-thread-4) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2861054 INFO  (jetty-launcher-2846-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2861054 INFO  (jetty-launcher-2846-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2861054 INFO  (jetty-launcher-2846-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2861054 INFO  (jetty-launcher-2846-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-11T04:52:23.489Z
   [junit4]   2> 2861135 INFO  (jetty-launcher-2846-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1b1691b2{/solr,null,AVAILABLE}
   [junit4]   2> 2861136 INFO  (jetty-launcher-2846-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@58e73dfa{SSL,[ssl, http/1.1]}{127.0.0.1:37108}
   [junit4]   2> 2861136 INFO  (jetty-launcher-2846-thread-3) [    ] o.e.j.s.Server Started @2867650ms
   [junit4]   2> 2861136 INFO  (jetty-launcher-2846-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37108}
   [junit4]   2> 2861136 ERROR (jetty-launcher-2846-thread-3) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2861136 INFO  (jetty-launcher-2846-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2861136 INFO  (jetty-launcher-2846-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2861136 INFO  (jetty-launcher-2846-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2861136 INFO  (jetty-launcher-2846-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-11T04:52:23.571Z
   [junit4]   2> 2861181 INFO  (jetty-launcher-2846-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@221ab55f{/solr,null,AVAILABLE}
   [junit4]   2> 2861181 INFO  (jetty-launcher-2846-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@472a2e73{SSL,[ssl, http/1.1]}{127.0.0.1:51654}
   [junit4]   2> 2861181 INFO  (jetty-launcher-2846-thread-1) [    ] o.e.j.s.Server Started @2867696ms
   [junit4]   2> 2861181 INFO  (jetty-launcher-2846-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51654}
   [junit4]   2> 2861181 ERROR (jetty-launcher-2846-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2861182 INFO  (jetty-launcher-2846-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2861182 INFO  (jetty-launcher-2846-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2861182 INFO  (jetty-launcher-2846-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2861182 INFO  (jetty-launcher-2846-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-11T04:52:23.617Z
   [junit4]   2> 2861184 INFO  (jetty-launcher-2846-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2861231 INFO  (jetty-launcher-2846-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2861241 INFO  (jetty-launcher-2846-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2861272 INFO  (jetty-launcher-2846-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42919/solr
   [junit4]   2> 2861286 INFO  (jetty-launcher-2846-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2861331 INFO  (jetty-launcher-2846-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42919/solr
   [junit4]   2> 2861356 INFO  (jetty-launcher-2846-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42919/solr
   [junit4]   2> 2861428 INFO  (jetty-launcher-2846-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42919/solr
   [junit4]   2> 2861535 INFO  (jetty-launcher-2846-thread-1) [n:127.0.0.1:51654_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2861555 INFO  (jetty-launcher-2846-thread-2) [n:127.0.0.1:38210_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2861562 INFO  (jetty-launcher-2846-thread-1) [n:127.0.0.1:51654_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51654_solr
   [junit4]   2> 2861563 INFO  (jetty-launcher-2846-thread-1) [n:127.0.0.1:51654_solr    ] o.a.s.c.Overseer Overseer (id=97772263390445577-127.0.0.1:51654_solr-n_0000000000) starting
   [junit4]   2> 2861568 INFO  (jetty-launcher-2846-thread-2) [n:127.0.0.1:38210_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38210_solr
   [junit4]   2> 2861570 INFO  (zkCallback-2865-thread-1-processing-n:127.0.0.1:51654_solr) [n:127.0.0.1:51654_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2861572 INFO  (zkCallback-2862-thread-1-processing-n:127.0.0.1:38210_solr) [n:127.0.0.1:38210_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2861581 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b5b5b0446000a, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2861589 INFO  (jetty-launcher-2846-thread-4) [n:127.0.0.1:45966_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2861591 INFO  (jetty-launcher-2846-thread-4) [n:127.0.0.1:45966_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2861598 INFO  (jetty-launcher-2846-thread-4) [n:127.0.0.1:45966_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45966_solr
   [junit4]   2> 2861599 INFO  (zkCallback-2865-thread-1-processing-n:127.0.0.1:51654_solr) [n:127.0.0.1:51654_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2861601 INFO  (zkCallback-2862-thread-1-processing-n:127.0.0.1:38210_solr) [n:127.0.0.1:38210_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2861632 INFO  (zkCallback-2867-thread-1-processing-n:127.0.0.1:45966_solr) [n:127.0.0.1:45966_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2861633 INFO  (jetty-launcher-2846-thread-1) [n:127.0.0.1:51654_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51654_solr
   [junit4]   2> 2861634 INFO  (zkCallback-2867-thread-1-processing-n:127.0.0.1:45966_solr) [n:127.0.0.1:45966_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2861634 INFO  (zkCallback-2862-thread-1-processing-n:127.0.0.1:38210_solr) [n:127.0.0.1:38210_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2861698 INFO  (zkCallback-2865-thread-2-processing-n:127.0.0.1:51654_solr) [n:127.0.0.1:51654_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2861714 INFO  (jetty-launcher-2846-thread-3) [n:127.0.0.1:37108_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 2861716 INFO  (jetty-launcher-2846-thread-3) [n:127.0.0.1:37108_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2861719 INFO  (jetty-launcher-2846-thread-3) [n:127.0.0.1:37108_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37108_solr
   [junit4]   2> 2861720 INFO  (zkCallback-2867-thread-1-processing-n:127.0.0.1:45966_solr) [n:127.0.0.1:45966_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2861720 INFO  (zkCallback-2862-thread-1-processing-n:127.0.0.1:38210_solr) [n:127.0.0.1:38210_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2861720 INFO  (zkCallback-2865-thread-1-processing-n:127.0.0.1:51654_solr) [n:127.0.0.1:51654_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2861721 INFO  (zkCallback-2871-thread-1-processing-n:127.0.0.1:37108_solr) [n:127.0.0.1:37108_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2861986 INFO  (jetty-launcher-2846-thread-2) [n:127.0.0.1:38210_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_3BBD3F23147BAA1F-001/tempDir-001/node2/.
   [junit4]   2> 2861988 INFO  (jetty-launcher-2846-thread-4) [n:127.0.0.1:45966_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_3BBD3F23147BAA1F-001/tempDir-001/node4/.
   [junit4]   2> 2862079 INFO  (jetty-launcher-2846-thread-1) [n:127.0.0.1:51654_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_3BBD3F23147BAA1F-001/tempDir-001/node1/.
   [junit4]   2> 2862159 INFO  (jetty-launcher-2846-thread-3) [n:127.0.0.1:37108_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_3BBD3F23147BAA1F-001/tempDir-001/node3/.
   [junit4]   2> 2865247 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[3BBD3F23147BAA1F]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 2865251 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[3BBD3F23147BAA1F]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42919/solr ready
   [junit4]   2> 2865363 INFO  (TEST-CollectionsAPIDistributedZkTest.testZeroNumShards-seed#[3BBD3F23147BAA1F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testZeroNumShards
   [junit4]   2> 2865364 INFO  (TEST-CollectionsAPIDistributedZkTest.testZeroNumShards-seed#[3BBD3F23147BAA1F]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 2865443 INFO  (qtp810562917-18674) [n:127.0.0.1:37108_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=10&collection.configName=conf&name=acollection&action=CREATE&numShards=0&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2865471 INFO  (OverseerThreadFactory-4682-thread-1-processing-n:127.0.0.1:51654_solr) [n:127.0.0.1:51654_solr    ] o.a.s.c.CreateCollectionCmd Create collection acollection
   [junit4]   2> 2865471 ERROR (OverseerThreadFactory-4682-thread-1-processing-n:127.0.0.1:51654_solr) [n:127.0.0.1:51654_solr    ] o.a.s.c.OverseerCollectionMessageHandler Collection: acollection operation: create failed:org.apache.solr.common.SolrException: numShards must be > 0
   [junit4]   2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:124)
   [junit4]   2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:226)
   [junit4]   2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:464)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>
   [junit4]   2> 2865486 INFO  (qtp810562917-18674) [n:127.0.0.1:37108_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=10&collection.configName=conf&name=acollection&action=CREATE&numShards=0&wt=javabin&version=2} status=400 QTime=43
   [junit4]   2> 2865487 INFO  (TEST-CollectionsAPIDistributedZkTest.testZeroNumShards-seed#[3BBD3F23147BAA1F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testZeroNumShards
   [junit4]   2> 2865508 INFO  (OverseerCollectionConfigSetProcessor-97772263390445577-127.0.0.1:51654_solr-n_0000000000) [n:127.0.0.1:51654_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2865552 INFO  (TEST-CollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection-seed#[3BBD3F23147BAA1F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting deletePartiallyCreatedCollection
   [junit4]   2> 2865553 INFO  (TEST-CollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection-seed#[3BBD3F23147BAA1F]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 2865567 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr    ] o.a.s.h.a.CoreAdminOperation core create command dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_3BBD3F23147BAA1F-001/tempDir-002&collection.configName=conf&name=halfdeletedcollection_shard1_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&wt=javabin&version=2
   [junit4]   2> 2865568 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2865570 INFO  (OverseerStateUpdate-97772263390445577-127.0.0.1:51654_solr-n_0000000000) [n:127.0.0.1:51654_solr    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 2866637 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2866646 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.s.IndexSchema [halfdeletedcollection_shard1_replica1] Schema name=minimal
   [junit4]   2> 2866659 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2866659 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection_shard1_replica1' using configuration from collection halfdeletedcollection, trusted=true
   [junit4]   2> 2866660 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2866660 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrCore [[halfdeletedcollection_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_3BBD3F23147BAA1F-001/tempDir-001/node3/halfdeletedcollection_shard1_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_3BBD3F23147BAA1F-001/tempDir-002/]
   [junit4]   2> 2866937 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2866938 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2866950 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2866951 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2866962 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@18930617[halfdeletedcollection_shard1_replica1] main]
   [junit4]   2> 2866963 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2866964 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2866964 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2866964 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1564356220707405824
   [junit4]   2> 2866965 INFO  (searcherExecutor-4691-thread-1-processing-n:127.0.0.1:37108_solr x:halfdeletedcollection_shard1_replica1 c:halfdeletedcollection) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrCore [halfdeletedcollection_shard1_replica1] Registered new searcher Searcher@18930617[halfdeletedcollection_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2867061 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2867061 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2867061 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:37108/solr/halfdeletedcollection_shard1_replica1/
   [junit4]   2> 2867061 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2867061 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:37108/solr/halfdeletedcollection_shard1_replica1/ has no replicas
   [junit4]   2> 2867061 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2867072 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:37108/solr/halfdeletedcollection_shard1_replica1/ shard2
   [junit4]   2> 2867179 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2867186 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr c:halfdeletedcollection   x:halfdeletedcollection_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_3BBD3F23147BAA1F-001/tempDir-002&collection.configName=conf&name=halfdeletedcollection_shard1_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&wt=javabin&version=2} status=0 QTime=1618
   [junit4]   2> 2867187 INFO  (qtp810562917-18672) [n:127.0.0.1:37108_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=halfdeletedcollection&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2867198 INFO  (OverseerThreadFactory-4682-thread-2-processing-n:127.0.0.1:51654_solr) [n:127.0.0.1:51654_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 2867243 INFO  (qtp810562917-18676) [n:127.0.0.1:37108_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.halfdeletedcollection.shard2.replica1, tag=null
   [junit4]   2> 2867244 INFO  (qtp810562917-18676) [n:127.0.0.1:37108_solr    ] o.a.s.c.SolrCore [halfdeletedcollection_shard1_replica1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1bfb3fe8
   [junit4]   2> 2867255 INFO  (qtp810562917-18676) [n:127.0.0.1:37108_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.halfdeletedcollection.shard2.replica1, tag=469450728
   [junit4]   2> 2867266 INFO  (qtp810562917-18676) [n:127.0.0.1:37108_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=halfdeletedcollection_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=22
   [junit4]   2> 2867973 INFO  (qtp810562917-18672) [n:127.0.0.1:37108_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=halfdeletedcollection&action=DELETE&wt=javabin&version=2} status=0 QTime=785
   [junit4]   2> 2867975 INFO  (qtp810562917-18677) [n:127.0.0.1:37108_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2867975 INFO  (qtp810562917-18677) [n:127.0.0.1:37108_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2867986 INFO  (OverseerCollectionConfigSetProcessor-97772263390445577-127.0.0.1:51654_solr-n_0000000000) [n:127.0.0.1:51654_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2867992 INFO  (qtp810562917-18677) [n:127.0.0.1:37108_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=halfdeletedcollection&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2868016 INFO  (OverseerThreadFactory-4682-thread-3-processing-n:127.0.0.1:51654_solr) [n:127.0.0.1:51654_solr    ] o.a.s.c.CreateCollectionCmd Create collection halfdeletedcollection
   [junit4]   2> 2868502 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=halfdeletedcollection_shard2_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&shard=shard2&wt=javabin&version=2
   [junit4]   2> 2868503 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2868503 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=halfdeletedcollection_shard1_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&shard=shard1&wt=javabin&version=2
   [junit4]   2> 2868503 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2868613 INFO  (zkCallback-2862-thread-1-processing-n:127.0.0.1:38210_solr) [n:127.0.0.1:38210_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2868614 INFO  (zkCallback-2865-thread-1-processing-n:127.0.0.1:51654_solr) [n:127.0.0.1:51654_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2869607 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2869611 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2869623 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.s.IndexSchema [halfdeletedcollection_shard1_replica1] Schema name=minimal
   [junit4]   2> 2869625 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.s.IndexSchema [halfdeletedcollection_shard2_replica1] Schema name=minimal
   [junit4]   2> 2869627 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2869627 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection_shard2_replica1' using configuration from collection halfdeletedcollection, trusted=true
   [junit4]   2> 2869630 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2869630 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection_shard1_replica1' using configuration from collection halfdeletedcollection, trusted=true
   [junit4]   2> 2869631 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2869631 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrCore [[halfdeletedcollection_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_3BBD3F23147BAA1F-001/tempDir-001/node2/halfdeletedcollection_shard1_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_3BBD3F23147BAA1F-001/tempDir-001/node2/./halfdeletedcollection_shard1_replica1/data/]
   [junit4]   2> 2869634 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2869634 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.SolrCore [[halfdeletedcollection_shard2_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_3BBD3F23147BAA1F-001/tempDir-001/node1/halfdeletedcollection_shard2_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_3BBD3F23147BAA1F-001/tempDir-001/node1/./halfdeletedcollection_shard2_replica1/data/]
   [junit4]   2> 2869914 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2869914 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2869915 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2869916 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2869924 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2869924 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2869970 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4e98523e[halfdeletedcollection_shard1_replica1] main]
   [junit4]   2> 2869971 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2869972 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2869972 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2869972 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1564356223861522432
   [junit4]   2> 2869974 INFO  (searcherExecutor-4697-thread-1-processing-n:127.0.0.1:38210_solr x:halfdeletedcollection_shard1_replica1 s:shard1 c:halfdeletedcollection) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrCore [halfdeletedcollection_shard1_replica1] Registered new searcher Searcher@4e98523e[halfdeletedcollection_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2869979 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2869979 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2869980 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:38210/solr/halfdeletedcollection_shard1_replica1/
   [junit4]   2> 2869981 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2869981 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:38210/solr/halfdeletedcollection_shard1_replica1/ has no replicas
   [junit4]   2> 2869981 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2869982 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2869982 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2869983 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3b06913e[halfdeletedcollection_shard2_replica1] main]
   [junit4]   2> 2869984 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2869985 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2869985 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2869985 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1564356223875153920
   [junit4]   2> 2869987 INFO  (searcherExecutor-4696-thread-1-processing-n:127.0.0.1:51654_solr x:halfdeletedcollection_shard2_replica1 s:shard2 c:halfdeletedcollection) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.SolrCore [halfdeletedcollection_shard2_replica1] Registered new searcher Searcher@3b06913e[halfdeletedcollection_shard2_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2869989 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:38210/solr/halfdeletedcollection_shard1_replica1/ shard1
   [junit4]   2> 2869995 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2869995 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2869995 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:51654/solr/halfdeletedcollection_shard2_replica1/
   [junit4]   2> 2869995 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2869995 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:51654/solr/halfdeletedcollection_shard2_replica1/ has no replicas
   [junit4]   2> 2869995 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2869999 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:51654/solr/halfdeletedcollection_shard2_replica1/ shard2
   [junit4]   2> 2870103 INFO  (zkCallback-2862-thread-1-processing-n:127.0.0.1:38210_solr) [n:127.0.0.1:38210_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2870105 INFO  (zkCallback-2865-thread-1-processing-n:127.0.0.1:51654_solr) [n:127.0.0.1:51654_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2870140 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2870142 INFO  (qtp1678053343-18652) [n:127.0.0.1:38210_solr c:halfdeletedcollection s:shard1  x:halfdeletedcollection_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=halfdeletedcollection_shard1_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&shard=shard1&wt=javabin&version=2} status=0 QTime=1641
   [junit4]   2> 2870150 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2870153 INFO  (qtp882061671-18683) [n:127.0.0.1:51654_solr c:halfdeletedcollection s:shard2  x:halfdeletedcollection_shard2_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=halfdeletedcollection_shard2_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&shard=shard2&wt=javabin&version=2} status=0 QTime=1651
   [junit4]   2> 2870161 INFO  (qtp810562917-18677) [n:127.0.0.1:37108_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 2870266 INFO  (zkCallback-2862-thread-1-processing-n:127.0.0.1:38210_solr) [n:127.0.0.1:38210_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2870267 INFO  (zkCallback-2865-thread-2-processing-n:127.0.0.1:51654_solr) [n:127.0.0.1:51654_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2871162 INFO  (qtp810562917-18677) [n:127.0.0.1:37108_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=halfdeletedcollection&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=3169
   [junit4]   2> 2871162 INFO  (qtp810562917-18678) [n:127.0.0.1:37108_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2871163 INFO  (qtp810562917-18678) [n:127.0.0.1:37108_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2871163 INFO  (TEST-CollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection-seed#[3BBD3F23147BAA1F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending deletePartiallyCreatedCollection
   [junit4]   2> 2871298 INFO  (TEST-CollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[3BBD3F23147BAA1F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testTooManyReplicas
   [junit4]   2> 2871299 INFO  (TEST-CollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[3BBD3F23147BAA1F]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 2871300 INFO  (qtp810562917-18674) [n:127.0.0.1:37108_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=halfdeletedcollection&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2871312 INFO  (OverseerCollectionConfigSetProcessor-97772263390445577-127.0.0.1:51654_solr-n_0000000000) [n:127.0.0.1:51654_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2871313 INFO  (OverseerThreadFactory-4682-thread-4-processing-n:127.0.0.1:51654_solr) [n:127.0.0.1:51654_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 2871317 INFO  (qtp1678053343-18653) [n:127.0.0.1:38210_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.halfdeletedcollection.shard1.replica1, tag=null
   [junit4]   2> 2871318 INFO  (qtp1678053343-18653) [n:127.0.0.1:38210_solr    ] o.a.s.c.SolrCore [halfdeletedcollection_shard1_replica1]  CLOSING SolrCore org.apache.solr.core.SolrCore@22720718
   [junit4]   2> 2871319 INFO  (qtp882061671-18684) [n:127.0.0.1:51654_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.halfdeletedcollection.shard2.replica1, tag=null
   [junit4]   2> 2871422 INFO  (qtp882061671-18684) [n:127.0.0.1:51654_solr    ] o.a.s.c.SolrCore [halfdeletedcollection_shard2_replica1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4915efc4
   [junit4]   2> 2871441 INFO  (qtp1678053343-18653) [n:127.0.0.1:38210_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.halfdeletedcollection.shard1.replica1, tag=577898264
   [junit4]   2> 2871454 INFO  (qtp1678053343-18653) [n:127.0.0.1:38210_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=halfdeletedcollection_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=137
   [junit4]   2> 2871455 INFO  (qtp882061671-18684) [n:127.0.0.1:51654_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.halfdeletedcollection.shard2.replica1, tag=1226174404
   [junit4]   2> 2871462 INFO  (qtp882061671-18684) [n:127.0.0.1:51654_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=halfdeletedcollection_shard2_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=142
   [junit4]   2> 2872167 INFO  (qtp810562917-18674) [n:127.0.0.1:37108_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=halfdeletedcollection&action=DELETE&wt=javabin&version=2} status=0 QTime=866
   [junit4]   2> 2872169 INFO  (qtp810562917-18676) [n:127.0.0.1:37108_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2872180 INFO  (OverseerThreadFactory-4682-thread-5-processing-n:127.0.0.1:51654_solr) [n:127.0.0.1:51654_solr    ] o.a.s.c.CreateCollectionCmd Create collection collection
   [junit4]   2> 2872180 INFO  (OverseerCollectionConfigSetProcessor-97772263390445577-127.0.0.1:51654_solr-n_0000000000) [n:127.0.0.1:51654_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2872180 WARN  (OverseerThreadFactory-4682-thread-5-processing-n:127.0.0.1:51654_solr) [n:127.0.0.1:51654_solr    ] o.a.s.c.CreateCollectionCmd Specified replicationFactor of 10 on collection collection is higher than or equal to the number of Solr instances currently live or live and part of your createNodeSet(4). It's unusual to run two replica of the same slice on the same Solr-instance.
   [junit4]   2> 2872181 ERROR (OverseerThreadFactory-4682-thread-5-processing-n:127.0.0.1:51654_solr) [n:127.0.0.1:51654_solr    ] o.a.s.c.OverseerCollectionMessageHandler Collection: collection operation: create failed:org.apache.solr.common.SolrException: Cannot create collection collection. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 4. This allows a maximum of 4 to be created. Value of numShards is 2 and value of replicationFactor is 10. This requires 20 shards to be created (higher than the allowed number)
   [junit4]   2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:155)
   [junit4]   2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:226)
   [junit4]   2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:464)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>
   [junit4]   2> 2872186 INFO  (qtp810562917-18676) [n:127.0.0.1:37108_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2} status=400 QTime=17
   [junit4]   2> 2872186 INFO  (TEST-CollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[3BBD3F23147BAA1F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testTooManyReplicas
   [junit4]   2> 2872253 INFO  (TEST-CollectionsAPIDistributedZkTest.testCollectionsAPI-seed#[3BBD3F23147BAA1F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCollectionsAPI
   [junit4]   2> 2872254 INFO  (TEST-CollectionsAPIDistributedZkTest.testCollectionsAPI-seed#[3BBD3F23147BAA1F]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 2872256 INFO  (qtp810562917-18672) [n:127.0.0.1:37108_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=e2b326e2-4beb-4f6c-a911-73a14a157052&replicationFactor=3&collection.configName=conf2&maxShardsPerNode=4&name=awhollynewcollection_0&action=CREATE&numShards=4&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2872257 INFO  (qtp810562917-18672) [n:127.0.0.1:37108_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=e2b326e2-4beb-4f6c-a911-73a14a157052&replicationFactor=3&collection.configName=conf2&maxShardsPerNode=4&name=awhollynewcollection_0&action=CREATE&numShards=4&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 2872268 INFO  (OverseerThreadFactory-4682-thread-5-processing-n:127.0.0.1:51654_solr) [n:127.0.0.1:51654_solr    ] o.a.s.c.CreateCollectionCmd Create collection awhollynewcollection_0
   [junit4]   2> 2872268 INFO  (OverseerCollectionConfigSetProcessor-97772263390445577-127.0.0.1:51654_solr-n_0000000000) [n:127.0.0.1:51654_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2872484 INFO  (zkCallback-2876-thread-5) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/awhollynewcollection_0/state.json] for collection [awhollynewcollection_0] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2872664 INFO  (qtp1678053343-18654) [n:127.0.0.1:38210_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598600782715&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard2_replica1&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard2&wt=javabin&version=2} status=0 QTime=7
   [junit4]   2> 2872668 INFO  (parallelCoreAdminExecutor-4683-thread-1-processing-n:127.0.0.1:38210_solr e2b326e2-4beb-4f6c-a911-73a14a15705230262598600782715 CREATE) [n:127.0.0.1:38210_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598600782715&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard2_replica1&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard2&wt=javabin&version=2
   [junit4]   2> 2872672 INFO  (qtp882061671-18685) [n:127.0.0.1:51654_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598600685873&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard3_replica1&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard3&wt=javabin&version=2} status=0 QTime=86
   [junit4]   2> 2872673 INFO  (parallelCoreAdminExecutor-4686-thread-1-processing-n:127.0.0.1:51654_solr e2b326e2-4beb-4f6c-a911-73a14a15705230262598600685873 CREATE) [n:127.0.0.1:51654_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598600685873&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard3_replica1&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard3&wt=javabin&version=2
   [junit4]   2> 2872715 INFO  (qtp810562917-18675) [n:127.0.0.1:37108_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598624111855&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard1_replica1&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard1&wt=javabin&version=2} status=0 QTime=102
   [junit4]   2> 2872740 INFO  (parallelCoreAdminExecutor-4688-thread-1-processing-n:127.0.0.1:37108_solr e2b326e2-4beb-4f6c-a911-73a14a15705230262598624111855 CREATE) [n:127.0.0.1:37108_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598624111855&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard1_replica1&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard1&wt=javabin&version=2
   [junit4]   2> 2872776 INFO  (qtp882061671-18682) [n:127.0.0.1:51654_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598625970927&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard1_replica3&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard1&wt=javabin&version=2} status=0 QTime=148
   [junit4]   2> 2872804 INFO  (qtp882061671-18684) [n:127.0.0.1:51654_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598616108132&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard4_replica2&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard4&wt=javabin&version=2} status=0 QTime=68
   [junit4]   2> 2872805 INFO  (parallelCoreAdminExecutor-4686-thread-3-processing-n:127.0.0.1:51654_solr e2b326e2-4beb-4f6c-a911-73a14a15705230262598616108132 CREATE) [n:127.0.0.1:51654_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598616108132&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard4_replica2&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard4&wt=javabin&version=2
   [junit4]   2> 2872824 INFO  (qtp1678053343-18647) [n:127.0.0.1:38210_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598600834088&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard3_replica2&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard3&wt=javabin&version=2} status=0 QTime=87
   [junit4]   2> 2872825 INFO  (parallelCoreAdminExecutor-4683-thread-3-processing-n:127.0.0.1:38210_solr e2b326e2-4beb-4f6c-a911-73a14a15705230262598600834088 CREATE) [n:127.0.0.1:38210_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598600834088&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard3_replica2&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard3&wt=javabin&version=2
   [junit4]   2> 2872825 INFO  (qtp810562917-18678) [n:127.0.0.1:37108_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598664104019&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard3_replica3&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard3&wt=javabin&version=2} status=0 QTime=139
   [junit4]   2> 2872826 INFO  (parallelCoreAdminExecutor-4688-thread-3-processing-n:127.0.0.1:37108_solr e2b326e2-4beb-4f6c-a911-73a14a15705230262598664104019 CREATE) [n:127.0.0.1:37108_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598664104019&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard3_replica3&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard3&wt=javabin&version=2
   [junit4]   2> 2872840 INFO  (qtp1678053343-18651) [n:127.0.0.1:38210_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598608103697&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard4_replica3&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard4&wt=javabin&version=2} status=0 QTime=176
   [junit4]   2> 2872840 INFO  (parallelCoreAdminExecutor-4683-thread-2-processing-n:127.0.0.1:38210_solr e2b326e2-4beb-4f6c-a911-73a14a15705230262598608103697 CREATE) [n:127.0.0.1:38210_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598608103697&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard4_replica3&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard4&wt=javabin&version=2
   [junit4]   2> 2872912 INFO  (qtp810562917-18677) [n:127.0.0.1:37108_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598632103422&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard2_replica2&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard2&wt=javabin&version=2} status=0 QTime=271
   [junit4]   2> 2872913 INFO  (parallelCoreAdminExecutor-4688-thread-2-processing-n:127.0.0.1:37108_solr e2b326e2-4beb-4f6c-a911-73a14a15705230262598632103422 CREATE) [n:127.0.0.1:37108_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598632103422&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard2_replica2&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard2&wt=javabin&version=2
   [junit4]   2> 2872916 INFO  (qtp397407894-18666) [n:127.0.0.1:45966_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598649805408&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard4_replica1&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard4&wt=javabin&version=2} status=0 QTime=124
   [junit4]   2> 2872917 INFO  (parallelCoreAdminExecutor-4686-thread-2-processing-n:127.0.0.1:51654_solr e2b326e2-4beb-4f6c-a911-73a14a15705230262598625970927 CREATE) [n:127.0.0.1:51654_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598625970927&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard1_replica3&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard1&wt=javabin&version=2
   [junit4]   2> 2872959 INFO  (qtp397407894-18668) [n:127.0.0.1:45966_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598660125484&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard1_replica2&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard1&wt=javabin&version=2} status=0 QTime=5
   [junit4]   2> 2872960 INFO  (parallelCoreAdminExecutor-4684-thread-1-processing-n:127.0.0.1:45966_solr e2b326e2-4beb-4f6c-a911-73a14a15705230262598649805408 CREATE) [n:127.0.0.1:45966_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598649805408&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard4_replica1&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard4&wt=javabin&version=2
   [junit4]   2> 2872962 INFO  (parallelCoreAdminExecutor-4684-thread-1-processing-n:127.0.0.1:45966_solr e2b326e2-4beb-4f6c-a911-73a14a15705230262598649805408 CREATE) [n:127.0.0.1:45966_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2872973 INFO  (parallelCoreAdminExecutor-4684-thread-2-processing-n:127.0.0.1:45966_solr e2b326e2-4beb-4f6c-a911-73a14a15705230262598660125484 CREATE) [n:127.0.0.1:45966_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598660125484&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard1_replica2&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard1&wt=javabin&version=2
   [junit4]   2> 2873001 INFO  (qtp397407894-18666) [n:127.0.0.1:45966_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598612099620&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard2_replica3&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard2&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2873002 INFO  (parallelCoreAdminExecutor-4684-thread-3-processing-n:127.0.0.1:45966_solr e2b326e2-4beb-4f6c-a911-73a14a15705230262598612099620 CREATE) [n:127.0.0.1:45966_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=e2b326e2-4beb-4f6c-a911-73a14a15705230262598612099620&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard2_replica3&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard2&wt=javabin&version=2
   [junit4]   2> 2873003 INFO  (qtp397407894-18661) [n:127.0.0.1:45966_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : e2b326e2-4beb-4f6c-a911-73a14a15705230262598660125484
   [junit4]   2> 2873003 INFO  (qtp397407894-18661) [n:127.0.0.1:

[...truncated too long message...]

.java:463)
   [junit4]   2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:213)
   [junit4]   2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:594)
   [junit4]   2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:171)
   [junit4]   2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
   [junit4]   2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
   [junit4]   2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
   [junit4]   2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:204)
   [junit4]   2> at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:77)
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
   [junit4]   2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
   [junit4]   2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
   [junit4]   2> ... 53 more
   [junit4]   2> 5703871 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[3BBD3F23147BAA1F]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44105 44105
   [junit4]   2> 5703871 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[3BBD3F23147BAA1F]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_3BBD3F23147BAA1F-001
   [junit4]   2> Apr 11, 2017 5:39:46 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 18 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false): {}, locale=th-TH-u-nu-thai-x-lvariant-TH, timezone=NZ
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=2,free=272222752,total=525336576
   [junit4]   2> NOTE: All tests run in this JVM: [MBeansHandlerTest, TestDefaultSearchFieldResource, TestDocSet, SimpleMLTQParserTest, SolrIndexConfigTest, TestConfig, TestExtendedDismaxParser, OutputWriterTest, TestCorePropertiesReload, TestGroupingSearch, TestRecovery, PrimitiveFieldTypeTest, TestSQLHandler, TestQuerySenderNoQuery, TestPKIAuthenticationPlugin, TestFileDictionaryLookup, TestTolerantSearch, TestReload, OutOfBoxZkACLAndCredentialsProvidersTest, TestCloudPseudoReturnFields, SolrMetricManagerTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, StressHdfsTest, TestLeaderInitiatedRecoveryThread, VMParamsZkACLAndCredentialsProvidersTest, TestPHPSerializedResponseWriter, TestManagedSchemaThreadSafety, TestZkChroot, SolrIndexMetricsTest, TestPivotHelperCode, DocValuesMultiTest, SortSpecParsingTest, RecoveryZkTest, SecurityConfHandlerTest, TestReplicationHandlerBackup, TestFieldCacheSortRandom, LeaderElectionTest, TestTolerantUpdateProcessorRandomCloud, LargeFieldTest, CurrencyFieldXmlFileTest, TestImplicitCoreProperties, TestSchemalessBufferedUpdates, TestIBSimilarityFactory, TestSubQueryTransformer, StatsComponentTest, SuggesterTest, SpellCheckCollatorWithCollapseTest, CoreAdminCreateDiscoverTest, RestartWhileUpdatingTest, HighlighterConfigTest, TestShortCircuitedRequests, DebugComponentTest, TestSolrQueryParserDefaultOperatorResource, HardAutoCommitTest, BigEndianAscendingWordDeserializerTest, TestCloudManagedSchema, RemoteQueryErrorTest, TestConfigSetImmutable, TestCollectionAPI, TestReversedWildcardFilterFactory, TermVectorComponentDistributedTest, MigrateRouteKeyTest, HdfsNNFailoverTest, PreAnalyzedFieldManagedSchemaCloudTest, FullSolrCloudDistribCmdsTest, BJQParserTest, FieldAnalysisRequestHandlerTest, TestExactStatsCache, AnalyticsMergeStrategyTest, TestComponentsName, BasicDistributedZk2Test, TestRawTransformer, TestMiniSolrCloudCluster, RequestHandlersTest, TestCoreDiscovery, TestSolrCLIRunExample, TestNoOpRegenerator, FacetPivotSmallTest, QueryResultKeyTest, DistributedQueryComponentCustomSortTest, SolrCoreCheckLockOnStartupTest, TestSolrCloudWithKerberosAlt, BlockJoinFacetSimpleTest, DateFieldTest, TestSolrDeletionPolicy1, TestApiFramework, TestMinMaxOnMultiValuedField, OverseerTaskQueueTest, SubstringBytesRefFilterTest, TestManagedResource, TestSolrQueryParser, TestSimpleQParserPlugin, TestMaxScoreQueryParser, LukeRequestHandlerTest, DistributedFacetExistsSmallTest, DeleteShardTest, SharedFSAutoReplicaFailoverUtilsTest, TestJavabinTupleStreamParser, JsonValidatorTest, TestSchemaSimilarityResource, SolrSlf4jReporterTest, TestSolrJ, TestRandomCollapseQParserPlugin, TestComplexPhraseLeadingWildcard, DistributedVersionInfoTest, StatelessScriptUpdateProcessorFactoryTest, SimpleFacetsTest, AsyncCallRequestStatusResponseTest, TestStressUserVersions, TestCollationField, TestFieldTypeResource, RAMDirectoryFactoryTest, TestBlobHandler, UpdateParamsTest, OverseerTest, UninvertDocValuesMergePolicyTest, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, LeaderElectionIntegrationTest, TestReplicationHandler, AutoCommitTest, TestHashPartitioner, DistributedSpellCheckComponentTest, TestStressReorder, TestStressVersions, TestRangeQuery, TestQuerySenderListener, TestSolrIndexConfig, CopyFieldTest, ResponseLogComponentTest, BadComponentTest, TestMergePolicyConfig, TestSolrDeletionPolicy2, TestBinaryField, TestElisionMultitermQuery, ExternalFileFieldSortTest, TestSolrCoreProperties, TestPostingsSolrHighlighter, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestPhraseSuggestions, TestCharFilters, SynonymTokenizerTest, EchoParamsTest, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, TestJmxMonitoredMap, ResourceLoaderTest, TestFastOutputStream, URLClassifyProcessorTest, PreAnalyzedFieldTest, PrimUtilsTest, SpellingQueryConverterTest, DOMUtilTest, TestUtils, SliceStateTest, SystemInfoHandlerTest, UUIDFieldTest, CircularListTest, DistributedMLTComponentTest, CursorPagingTest, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestSimpleTrackingShardHandler, TestEmbeddedSolrServerSchemaAPI, ActionThrottleTest, AliasIntegrationTest, AssignTest, BaseCdcrDistributedZkTest, CdcrBootstrapTest, CloudExitableDirectoryReaderTest, CollectionTooManyReplicasTest, CollectionsAPISolrJTest, DeleteLastCustomShardedReplicaTest, DeleteNodeTest, DeleteStatusTest, DistribCursorPagingTest, DocValuesNotIndexedTest, LeaderFailoverAfterPartitionTest, MoveReplicaTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerCollectionConfigSetProcessorTest, OverseerModifyCollectionTest, SaslZkACLProviderTest, ShardSplitTest, HdfsChaosMonkeyNothingIsSafeTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [540/704 (4!)] on J2 in 149.18s, 21 tests, 1 failure <<< FAILURES!

[...truncated 43816 lines...]




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1280 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1280/

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 4 object(s) that were not released!!! [SolrCore, InternalHttpClient, MockDirectoryWrapper, MockDirectoryWrapper] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1025)  at org.apache.solr.core.SolrCore.reload(SolrCore.java:648)  at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1166)  at org.apache.solr.handler.IndexFetcher.lambda$reloadCore$0(IndexFetcher.java:877)  at java.lang.Thread.run(Thread.java:745)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:298)  at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:218)  at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:252)  at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1209)  at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:695)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:973)  at org.apache.solr.core.SolrCore.reload(SolrCore.java:648)  at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1166)  at org.apache.solr.handler.IndexFetcher.lambda$reloadCore$0(IndexFetcher.java:877)  at java.lang.Thread.run(Thread.java:745)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)  at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:461)  at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:326)  at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:415)  at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1179)  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:745)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)  at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:359)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:721)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:940)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:975)  at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:595)  at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:745)  

Stack Trace:
java.lang.AssertionError: ObjectTracker found 4 object(s) that were not released!!! [SolrCore, InternalHttpClient, MockDirectoryWrapper, MockDirectoryWrapper]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
        at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1025)
        at org.apache.solr.core.SolrCore.reload(SolrCore.java:648)
        at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1166)
        at org.apache.solr.handler.IndexFetcher.lambda$reloadCore$0(IndexFetcher.java:877)
        at java.lang.Thread.run(Thread.java:745)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
        at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)
        at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:298)
        at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:218)
        at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:252)
        at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1209)
        at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:695)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:973)
        at org.apache.solr.core.SolrCore.reload(SolrCore.java:648)
        at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1166)
        at org.apache.solr.handler.IndexFetcher.lambda$reloadCore$0(IndexFetcher.java:877)
        at java.lang.Thread.run(Thread.java:745)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
        at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
        at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:461)
        at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:326)
        at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:415)
        at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1179)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
        at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
        at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:359)
        at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:721)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:940)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:975)
        at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:595)
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)


        at __randomizedtesting.SeedInfo.seed([B35B4390D1EAEB95]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:302)
        at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12627 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/init-core-data-001
   [junit4]   2> 2749032 WARN  (SUITE-TestReplicationHandler-seed#[B35B4390D1EAEB95]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=120 numCloses=120
   [junit4]   2> 2749032 INFO  (SUITE-TestReplicationHandler-seed#[B35B4390D1EAEB95]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields
   [junit4]   2> 2749034 INFO  (SUITE-TestReplicationHandler-seed#[B35B4390D1EAEB95]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
   [junit4]   2> 2749040 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.SolrTestCaseJ4 ###Starting doTestStopPoll
   [junit4]   2> 2749040 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/solr-instance-001/collection1
   [junit4]   2> 2749044 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2749050 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@40aae145{/solr,null,AVAILABLE}
   [junit4]   2> 2749051 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@dd450b4{HTTP/1.1,[http/1.1]}{127.0.0.1:60275}
   [junit4]   2> 2749051 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.e.j.s.Server Started @2756874ms
   [junit4]   2> 2749051 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=60275}
   [junit4]   2> 2749051 ERROR (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2749051 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2749051 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 2749051 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2749051 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-12T13:46:15.713Z
   [junit4]   2> 2749052 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/solr-instance-001/solr.xml
   [junit4]   2> 2749058 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3d7e9c75, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 2749100 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d7e9c75
   [junit4]   2> 2749106 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d7e9c75
   [junit4]   2> 2749106 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d7e9c75
   [junit4]   2> 2749113 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/solr-instance-001/.
   [junit4]   2> 2749113 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 2749178 WARN  (coreLoadExecutor-6652-thread-1) [    x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2749178 INFO  (coreLoadExecutor-6652-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2749188 INFO  (coreLoadExecutor-6652-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2749191 INFO  (coreLoadExecutor-6652-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
   [junit4]   2> 2749191 INFO  (coreLoadExecutor-6652-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/solr-instance-001/./collection1, trusted=true
   [junit4]   2> 2749192 INFO  (coreLoadExecutor-6652-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d7e9c75
   [junit4]   2> 2749192 INFO  (coreLoadExecutor-6652-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2749192 INFO  (coreLoadExecutor-6652-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 2749194 INFO  (coreLoadExecutor-6652-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=96.0390625, floorSegmentMB=1.7890625, forceMergeDeletesPctAllowed=8.654817571753773, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 2749247 INFO  (coreLoadExecutor-6652-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2749247 INFO  (coreLoadExecutor-6652-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2749248 INFO  (coreLoadExecutor-6652-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=1.9873046875, floorSegmentMB=1.208984375, forceMergeDeletesPctAllowed=20.77780318045224, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 2749249 INFO  (coreLoadExecutor-6652-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7df53871[collection1] main]
   [junit4]   2> 2749249 INFO  (coreLoadExecutor-6652-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/solr-instance-001/collection1/conf
   [junit4]   2> 2749250 INFO  (coreLoadExecutor-6652-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 2749250 INFO  (coreLoadExecutor-6652-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2749251 INFO  (searcherExecutor-6653-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7df53871[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2749680 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/solr-instance-002/collection1
   [junit4]   2> 2749683 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2749684 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@23ce86d2{/solr,null,AVAILABLE}
   [junit4]   2> 2749684 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@d9c0a25{HTTP/1.1,[http/1.1]}{127.0.0.1:37580}
   [junit4]   2> 2749685 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.e.j.s.Server Started @2757508ms
   [junit4]   2> 2749685 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=37580}
   [junit4]   2> 2749685 ERROR (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2749685 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2749685 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 2749685 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2749685 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-12T13:46:16.347Z
   [junit4]   2> 2749686 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/solr-instance-002/solr.xml
   [junit4]   2> 2749691 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3d7e9c75, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 2749733 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d7e9c75
   [junit4]   2> 2749739 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d7e9c75
   [junit4]   2> 2749739 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d7e9c75
   [junit4]   2> 2749741 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/solr-instance-002/.
   [junit4]   2> 2749741 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[B35B4390D1EAEB95]) [    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 2749756 WARN  (coreLoadExecutor-6662-thread-1) [    x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2749757 INFO  (coreLoadExecutor-6662-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2749765 INFO  (coreLoadExecutor-6662-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2749768 INFO  (coreLoadExecutor-6662-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
   [junit4]   2> 2749768 INFO  (coreLoadExecutor-6662-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/solr-instance-002/./collection1, trusted=true
   [junit4]   2> 2749768 INFO  (coreLoadExecutor-6662-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d7e9c75
   [junit4]   2> 2749768 INFO  (coreLoadExecutor-6662-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2749768 INFO  (coreLoadExecutor-6662-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/solr-instance-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/solr-instance-002/./collection1/data/]
   [junit4]   2> 2749770 INFO  (coreLoadExecutor-6662-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=96.0390625, floorSegmentMB=1.7890625, forceMergeDeletesPctAllowed=8.654817571753773, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 2749831 INFO  (coreLoadExecutor-6662-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2749831 INFO  (coreLoadExecutor-6662-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2749832 INFO  (coreLoadExecutor-6662-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=1.9873046875, floorSegmentMB=1.208984375, forceMergeDeletesPctAllowed=20.77780318045224, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 2749833 INFO  (coreLoadExecutor-6662-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@761d4374[collection1] main]
   [junit4]   2> 2749833 INFO  (coreLoadExecutor-6662-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_B35B4390D1EAEB95-001/solr-instance-002/collection1/conf
   [junit4]   2> 2749841 INFO  (coreLoadExecutor-6662-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 2749842 INFO  (searcherExecutor-6663-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@761d4374[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2749845 INFO  (qtp2120876439-23802) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 2749846 INFO  (indexFetcher-6667-thread-1) [    x:collection1] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 2749846 INFO  (indexFetcher-6667-thread-1) [    x:collection1] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 2749846 INFO  (indexFetcher-6667-thread-1) [    x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 2749846 INFO  (indexFetcher-6667-thread-1) [    x:collection1] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 2750271 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/select params={q=*:*&sort=id+desc&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 2750273 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 1
   [junit4]   2> 2750273 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 0
   [junit4]   2> 2750274 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
   [junit4]   2> 2750274 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[3]} 0 0
   [junit4]   2> 2750275 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[4]} 0 0
   [junit4]   2> 2750275 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[5]} 0 0
   [junit4]   2> 2750275 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[6]} 0 0
   [junit4]   2> 2750276 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[7]} 0 0
   [junit4]   2> 2750276 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[8]} 0 0
   [junit4]   2> 2750277 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[9]} 0 0
   [junit4]   2> 2750277 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[10]} 0 0
   [junit4]   2> 2750277 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[11]} 0 0
   [junit4]   2> 2750278 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[12]} 0 0
   [junit4]   2> 2750278 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[13]} 0 0
   [junit4]   2> 2750279 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[14]} 0 0
   [junit4]   2> 2750279 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[15]} 0 0
   [junit4]   2> 2750279 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[16]} 0 0
   [junit4]   2> 2750280 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[17]} 0 0
   [junit4]   2> 2750280 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[18]} 0 0
   [junit4]   2> 2750281 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[19]} 0 0
   [junit4]   2> 2750281 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[20]} 0 0
   [junit4]   2> 2750281 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[21]} 0 0
   [junit4]   2> 2750282 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[22]} 0 0
   [junit4]   2> 2750282 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[23]} 0 0
   [junit4]   2> 2750283 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[24]} 0 0
   [junit4]   2> 2750283 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[25]} 0 0
   [junit4]   2> 2750283 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[26]} 0 0
   [junit4]   2> 2750284 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[27]} 0 0
   [junit4]   2> 2750284 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[28]} 0 0
   [junit4]   2> 2750285 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[29]} 0 0
   [junit4]   2> 2750285 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[30]} 0 0
   [junit4]   2> 2750285 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[31]} 0 0
   [junit4]   2> 2750286 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[32]} 0 0
   [junit4]   2> 2750286 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[33]} 0 0
   [junit4]   2> 2750287 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[34]} 0 0
   [junit4]   2> 2750287 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[35]} 0 0
   [junit4]   2> 2750287 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[36]} 0 0
   [junit4]   2> 2750288 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[37]} 0 0
   [junit4]   2> 2750288 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[38]} 0 0
   [junit4]   2> 2750289 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[39]} 0 0
   [junit4]   2> 2750289 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[40]} 0 0
   [junit4]   2> 2750289 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[41]} 0 0
   [junit4]   2> 2750290 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[42]} 0 0
   [junit4]   2> 2750290 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[43]} 0 0
   [junit4]   2> 2750291 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[44]} 0 0
   [junit4]   2> 2750291 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[45]} 0 0
   [junit4]   2> 2750292 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[46]} 0 0
   [junit4]   2> 2750292 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[47]} 0 0
   [junit4]   2> 2750292 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[48]} 0 0
   [junit4]   2> 2750293 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[49]} 0 0
   [junit4]   2> 2750293 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[50]} 0 0
   [junit4]   2> 2750294 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[51]} 0 0
   [junit4]   2> 2750294 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[52]} 0 0
   [junit4]   2> 2750294 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[53]} 0 0
   [junit4]   2> 2750295 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[54]} 0 0
   [junit4]   2> 2750295 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[55]} 0 0
   [junit4]   2> 2750296 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[56]} 0 0
   [junit4]   2> 2750296 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[57]} 0 0
   [junit4]   2> 2750297 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[58]} 0 0
   [junit4]   2> 2750297 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[59]} 0 0
   [junit4]   2> 2750297 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[60]} 0 0
   [junit4]   2> 2750323 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[61]} 0 25
   [junit4]   2> 2750325 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[62]} 0 0
   [junit4]   2> 2750325 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[63]} 0 0
   [junit4]   2> 2750326 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[64]} 0 0
   [junit4]   2> 2750326 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[65]} 0 0
   [junit4]   2> 2750327 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[66]} 0 0
   [junit4]   2> 2750327 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[67]} 0 0
   [junit4]   2> 2750329 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[68]} 0 0
   [junit4]   2> 2750330 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[69]} 0 0
   [junit4]   2> 2750332 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[70]} 0 0
   [junit4]   2> 2750332 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[71]} 0 0
   [junit4]   2> 2750333 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[72]} 0 0
   [junit4]   2> 2750333 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[73]} 0 0
   [junit4]   2> 2750334 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[74]} 0 0
   [junit4]   2> 2750334 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[75]} 0 0
   [junit4]   2> 2750335 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[76]} 0 0
   [junit4]   2> 2750335 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[77]} 0 0
   [junit4]   2> 2750335 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[78]} 0 0
   [junit4]   2> 2750336 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[79]} 0 0
   [junit4]   2> 2750336 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[80]} 0 0
   [junit4]   2> 2750337 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[81]} 0 0
   [junit4]   2> 2750337 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[82]} 0 0
   [junit4]   2> 2750337 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[83]} 0 0
   [junit4]   2> 2750338 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[84]} 0 0
   [junit4]   2> 2750338 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[85]} 0 0
   [junit4]   2> 2750339 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[86]} 0 0
   [junit4]   2> 2750339 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[87]} 0 0
   [junit4]   2> 2750339 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[88]} 0 0
   [junit4]   2> 2750340 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[89]} 0 0
   [junit4]   2> 2750340 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[90]} 0 0
   [junit4]   2> 2750341 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[91]} 0 0
   [junit4]   2> 2750341 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[92]} 0 0
   [junit4]   2> 2750341 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[93]} 0 0
   [junit4]   2> 2750342 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[94]} 0 0
   [junit4]   2> 2750342 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[95]} 0 0
   [junit4]   2> 2750343 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[96]} 0 0
   [junit4]   2> 2750343 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[97]} 0 0
   [junit4]   2> 2750344 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[98]} 0 0
   [junit4]   2> 2750344 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[99]} 0 0
   [junit4]   2> 2750344 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[100]} 0 0
   [junit4]   2> 2750345 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[101]} 0 0
   [junit4]   2> 2750345 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[102]} 0 0
   [junit4]   2> 2750346 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[103]} 0 0
   [junit4]   2> 2750346 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[104]} 0 0
   [junit4]   2> 2750347 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[105]} 0 0
   [junit4]   2> 2750347 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[106]} 0 0
   [junit4]   2> 2750347 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[107]} 0 0
   [junit4]   2> 2750348 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[108]} 0 0
   [junit4]   2> 2750348 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[109]} 0 0
   [junit4]   2> 2750349 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[110]} 0 0
   [junit4]   2> 2750349 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[111]} 0 0
   [junit4]   2> 2750349 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[112]} 0 0
   [junit4]   2> 2750350 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[113]} 0 0
   [junit4]   2> 2750350 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[114]} 0 0
   [junit4]   2> 2750351 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[115]} 0 0
   [junit4]   2> 2750351 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[116]} 0 0
   [junit4]   2> 2750352 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[117]} 0 0
   [junit4]   2> 2750352 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[118]} 0 0
   [junit4]   2> 2750352 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[119]} 0 0
   [junit4]   2> 2750353 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[120]} 0 0
   [junit4]   2> 2750353 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[121]} 0 0
   [junit4]   2> 2750354 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[122]} 0 0
   [junit4]   2> 2750354 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[123]} 0 0
   [junit4]   2> 2750354 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[124]} 0 0
   [junit4]   2> 2750355 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[125]} 0 0
   [junit4]   2> 2750355 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[126]} 0 0
   [junit4]   2> 2750356 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[127]} 0 0
   [junit4]   2> 2750356 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[128]} 0 0
   [junit4]   2> 2750357 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[129]} 0 0
   [junit4]   2> 2750357 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[130]} 0 0
   [junit4]   2> 2750358 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[131]} 0 0
   [junit4]   2> 2750358 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[132]} 0 0
   [junit4]   2> 2750358 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[133]} 0 0
   [junit4]   2> 2750359 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[134]} 0 0
   [junit4]   2> 2750359 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[135]} 0 0
   [junit4]   2> 2750360 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[136]} 0 0
   [junit4]   2> 2750360 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[137]} 0 0
   [junit4]   2> 2750360 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[138]} 0 0
   [junit4]   2> 2750361 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[139]} 0 0
   [junit4]   2> 2750361 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[140]} 0 0
   [junit4]   2> 2750362 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[141]} 0 0
   [junit4]   2> 2750362 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[142]} 0 0
   [junit4]   2> 2750363 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[143]} 0 0
   [junit4]   2> 2750363 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[144]} 0 0
   [junit4]   2> 2750363 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[145]} 0 0
   [junit4]   2> 2750364 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[146]} 0 0
   [junit4]   2> 2750364 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[147]} 0 0
   [junit4]   2> 2750365 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[148]} 0 0
   [junit4]   2> 2750365 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[149]} 0 0
   [junit4]   2> 2750365 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[150]} 0 0
   [junit4]   2> 2750366 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[151]} 0 0
   [junit4]   2> 2750366 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[152]} 0 0
   [junit4]   2> 2750367 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[153]} 0 0
   [junit4]   2> 2750367 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[154]} 0 0
   [junit4]   2> 2750367 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[155]} 0 0
   [junit4]   2> 2750368 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[156]} 0 0
   [junit4]   2> 2750368 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[157]} 0 0
   [junit4]   2> 2750369 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[158]} 0 0
   [junit4]   2> 2750369 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[159]} 0 0
   [junit4]   2> 2750369 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[160]} 0 0
   [junit4]   2> 2750370 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[161]} 0 0
   [junit4]   2> 2750370 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[162]} 0 0
   [junit4]   2> 2750371 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[163]} 0 0
   [junit4]   2> 2750371 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[164]} 0 0
   [junit4]   2> 2750371 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[165]} 0 0
   [junit4]   2> 2750372 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[166]} 0 0
   [junit4]   2> 2750372 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[167]} 0 0
   [junit4]   2> 2750373 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[168]} 0 0
   [junit4]   2> 2750373 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[169]} 0 0
   [junit4]   2> 2750373 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[170]} 0 0
   [junit4]   2> 2750374 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[171]} 0 0
   [junit4]   2> 2750374 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[172]} 0 0
   [junit4]   2> 2750375 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[173]} 0 0
   [junit4]   2> 2750375 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[174]} 0 0
   [junit4]   2> 2750375 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[175]} 0 0
   [junit4]   2> 2750376 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[176]} 0 0
   [junit4]   2> 2750376 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[177]} 0 0
   [junit4]   2> 2750377 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[178]} 0 0
   [junit4]   2> 2750377 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[179]} 0 0
   [junit4]   2> 2750377 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[180]} 0 0
   [junit4]   2> 2750378 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[181]} 0 0
   [junit4]   2> 2750378 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[182]} 0 0
   [junit4]   2> 2750379 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[183]} 0 0
   [junit4]   2> 2750379 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[184]} 0 0
   [junit4]   2> 2750379 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[185]} 0 0
   [junit4]   2> 2750380 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[186]} 0 0
   [junit4]   2> 2750380 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[187]} 0 0
   [junit4]   2> 2750381 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[188]} 0 0
   [junit4]   2> 2750381 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[189]} 0 0
   [junit4]   2> 2750381 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[190]} 0 0
   [junit4]   2> 2750382 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[191]} 0 0
   [junit4]   2> 2750382 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[192]} 0 0
   [junit4]   2> 2750383 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[193]} 0 0
   [junit4]   2> 2750383 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[194]} 0 0
   [junit4]   2> 2750384 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[195]} 0 0
   [junit4]   2> 2750384 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[196]} 0 0
   [junit4]   2> 2750384 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[197]} 0 0
   [junit4]   2> 2750385 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[198]} 0 0
   [junit4]   2> 2750385 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[199]} 0 0
   [junit4]   2> 2750386 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[200]} 0 0
   [junit4]   2> 2750386 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[201]} 0 0
   [junit4]   2> 2750386 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[202]} 0 0
   [junit4]   2> 2750387 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[203]} 0 0
   [junit4]   2> 2750387 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[204]} 0 0
   [junit4]   2> 2750388 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[205]} 0 0
   [junit4]   2> 2750388 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[206]} 0 0
   [junit4]   2> 2750389 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[207]} 0 0
   [junit4]   2> 2750389 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[208]} 0 0
   [junit4]   2> 2750389 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[209]} 0 0
   [junit4]   2> 2750390 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[210]} 0 0
   [junit4]   2> 2750390 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[211]} 0 0
   [junit4]   2> 2750391 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[212]} 0 0
   [junit4]   2> 2750391 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[213]} 0 0
   [junit4]   2> 2750391 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[214]} 0 0
   [junit4]   2> 2750392 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[215]} 0 0
   [junit4]   2> 2750392 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[216]} 0 0
   [junit4]   2> 2750393 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[217]} 0 0
   [junit4]   2> 2750393 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[218]} 0 0
   [junit4]   2> 2750393 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[219]} 0 0
   [junit4]   2> 2750394 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[220]} 0 0
   [junit4]   2> 2750394 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[221]} 0 0
   [junit4]   2> 2750395 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[222]} 0 0
   [junit4]   2> 2750395 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[223]} 0 0
   [junit4]   2> 2750395 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[224]} 0 0
   [junit4]   2> 2750396 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[225]} 0 0
   [junit4]   2> 2750396 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[226]} 0 0
   [junit4]   2> 2750397 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[227]} 0 0
   [junit4]   2> 2750397 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[228]} 0 0
   [junit4]   2> 2750398 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[229]} 0 0
   [junit4]   2> 2750398 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[230]} 0 0
   [junit4]   2> 2750398 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[231]} 0 0
   [junit4]   2> 2750399 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[232]} 0 0
   [junit4]   2> 2750399 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[233]} 0 0
   [junit4]   2> 2750400 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[234]} 0 0
   [junit4]   2> 2750400 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[235]} 0 0
   [junit4]   2> 2750400 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[236]} 0 0
   [junit4]   2> 2750401 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[237]} 0 0
   [junit4]   2> 2750401 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[238]} 0 0
   [junit4]   2> 2750402 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[239]} 0 0
   [junit4]   2> 2750402 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[240]} 0 0
   [junit4]   2> 2750402 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[241]} 0 0
   [junit4]   2> 2750403 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[242]} 0 0
   [junit4]   2> 2750403 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[243]} 0 0
   [junit4]   2> 2750403 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[244]} 0 0
   [junit4]   2> 2750404 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[245]} 0 0
   [junit4]   2> 2750404 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[246]} 0 0
   [junit4]   2> 2750405 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[247]} 0 0
   [junit4]   2> 2750405 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[248]} 0 0
   [junit4]   2> 2750405 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[249]} 0 0
   [junit4]   2> 2750406 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[250]} 0 0
   [junit4]   2> 2750406 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[251]} 0 0
   [junit4]   2> 2750407 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[252]} 0 0
   [junit4]   2> 2750407 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[253]} 0 0
   [junit4]   2> 2750407 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[254]} 0 0
   [junit4]   2> 2750413 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[255]} 0 0
   [junit4]   2> 2750414 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[256]} 0 0
   [junit4]   2> 2750415 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[257]} 0 0
   [junit4]   2> 2750415 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[258]} 0 0
   [junit4]   2> 2750416 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[259]} 0 0
   [junit4]   2> 2750416 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[260]} 0 0
   [junit4]   2> 2750417 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[261]} 0 0
   [junit4]   2> 2750417 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[262]} 0 0
   [junit4]   2> 2750417 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[263]} 0 0
   [junit4]   2> 2750418 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[264]} 0 0
   [junit4]   2> 2750418 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[265]} 0 0
   [junit4]   2> 2750419 INFO  (qtp2120876439-23806) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[266]} 0 0
   [junit4]   2> 2750419 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[267]} 0 0
   [junit4]   2> 2750419 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[268]} 0 0
   [junit4]   2> 2750420 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[269]} 0 0
   [junit4]   2> 2750420 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[270]} 0 0
   [junit4]   2> 2750420 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[271]} 0 0
   [junit4]   2> 2750421 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[272]} 0 0
   [junit4]   2> 2750421 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[273]} 0 0
   [junit4]   2> 2750422 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[274]} 0 0
   [junit4]   2> 2750422 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[275]} 0 0
   [junit4]   2> 2750422 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[276]} 0 0
   [junit4]   2> 2750423 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[277]} 0 0
   [junit4]   2> 2750423 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[278]} 0 0
   [junit4]   2> 2750424 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[279]} 0 0
   [junit4]   2> 2750424 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[280]} 0 0
   [junit4]   2> 2750424 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[281]} 0 0
   [junit4]   2> 2750425 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[282]} 0 0
   [junit4]   2> 2750425 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[283]} 0 0
   [junit4]   2> 2750426 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[284]} 0 0
   [junit4]   2> 2750426 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[285]} 0 0
   [junit4]   2> 2750426 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[286]} 0 0
   [junit4]   2> 2750427 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[287]} 0 0
   [junit4]   2> 2750427 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[288]} 0 0
   [junit4]   2> 2750428 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[289]} 0 0
   [junit4]   2> 2750428 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[290]} 0 0
   [junit4]   2> 2750429 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[291]} 0 0
   [junit4]   2> 2750429 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[292]} 0 0
   [junit4]   2> 2750429 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[293]} 0 0
   [junit4]   2> 2750430 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[294]} 0 0
   [junit4]   2> 2750430 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[295]} 0 0
   [junit4]   2> 2750431 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[296]} 0 0
   [junit4]   2> 2750431 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[297]} 0 0
   [junit4]   2> 2750432 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[298]} 0 0
   [junit4]   2> 2750432 INFO  (qtp2120876439-23804) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[299]} 0 0
   [junit4]   2> 2750432 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[300]} 0 0
   [junit4]   2> 2750433 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[301]} 0 0
   [junit4]   2> 2750433 INFO  (qtp2120876439-23803) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[302]} 0 0
   [junit4]   2> 2750434 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[303]} 0 0
   [junit4]   2> 2750434 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[304]} 0 0
   [junit4]   2> 2750434 INFO  (qtp2120876439-23800) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[305]} 0 0
   [junit4]   2> 2750435 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[306]} 0 0
   [junit4]   2> 2750435 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[307]} 0 0
   [junit4]   2> 2750436 INFO  (qtp2120876439-23799) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[308]} 0 0
   [junit4]   2> 2750436 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[309]} 0 0
   [junit4]   2> 2750437 INFO  (qtp2120876439-23805) [    x:collection1] o.a.s.u.p.LogUpdateProcesso

[...truncated too long message...]

t4]   2> SUITE-TestReplicationHandler-seed#[B35B4390D1EAEB95]:
   [junit4]   2>     java.lang.Object.wait(Native Method)
   [junit4]   2>     java.lang.Thread.join(Thread.java:1257)
   [junit4]   2>     [...com.carrotsearch.randomizedtesting.*]
   [junit4]   2>
   [junit4]   2> 3131341 INFO  (SUITE-TestReplicationHandler-seed#[B35B4390D1EAEB95]-worker) [    ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {name=PostingsFormat(name=LuceneVarGapFixedInterval), id=PostingsFormat(name=LuceneVarGapFixedInterval), newname=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{}, maxPointsInLeafNode=362, maxMBSortInHeap=5.859896226318396, sim=RandomSimilarity(queryNorm=true): {}, locale=no, timezone=Atlantic/Jan_Mayen
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=217440160,total=526909440
   [junit4]   2> NOTE: All tests run in this JVM: [HttpPartitionTest, TestTrieFacet, TestNumericTerms32, TestCoreDiscovery, DistributedQueryComponentOptimizationTest, CloudExitableDirectoryReaderTest, UUIDUpdateProcessorFallbackTest, TestOnReconnectListenerSupport, RulesTest, ResponseHeaderTest, TestOrdValues, TestGroupingSearch, OutputWriterTest, WrapperMergePolicyFactoryTest, AnalyticsQueryTest, ConcurrentDeleteAndCreateCollectionTest, DocValuesMultiTest, PKIAuthenticationIntegrationTest, CoreAdminHandlerTest, SpatialHeatmapFacetsTest, MergeStrategyTest, BadIndexSchemaTest, TestReversedWildcardFilterFactory, BigEndianAscendingWordSerializerTest, CdcrVersionReplicationTest, ChangedSchemaMergeTest, HdfsRestartWhileUpdatingTest, SimpleFacetsTest, TestDistributedMissingSort, LeaderFailureAfterFreshStartTest, GraphQueryTest, TestZkChroot, TestDocSet, SegmentsInfoRequestHandlerTest, FieldMutatingUpdateProcessorTest, TestCorePropertiesReload, TestPHPSerializedResponseWriter, ShowFileRequestHandlerTest, TestClassicSimilarityFactory, HdfsLockFactoryTest, TestStressReorder, TestConfigSetImmutable, TestRuleBasedAuthorizationPlugin, TestTrackingShardHandlerFactory, XmlUpdateRequestHandlerTest, AutoCommitTest, TestConfig, SolrMetricReporterTest, DeleteShardTest, MultiThreadedOCPTest, TestSystemCollAutoCreate, TestLMDirichletSimilarityFactory, OutOfBoxZkACLAndCredentialsProvidersTest, TestDocBasedVersionConstraints, TestSubQueryTransformerCrossCore, TestPushWriter, TestNRTOpen, AsyncCallRequestStatusResponseTest, TestDynamicFieldCollectionResource, TestShortCircuitedRequests, TestInitQParser, TestDistributedStatsComponentCardinality, SharedFSAutoReplicaFailoverUtilsTest, TemplateUpdateProcessorTest, HighlighterConfigTest, OverseerModifyCollectionTest, TestManagedSynonymFilterFactory, DirectUpdateHandlerTest, DebugComponentTest, HdfsChaosMonkeyNothingIsSafeTest, AddBlockUpdateTest, TestMiniSolrCloudCluster, TestSchemaNameResource, CollectionTooManyReplicasTest, DirectUpdateHandlerOptimizeTest, TestSolrFieldCacheBean, TestQuerySenderNoQuery, SynonymTokenizerTest, TestTolerantUpdateProcessorRandomCloud, TestPhraseSuggestions, TestHdfsCloudBackupRestore, TestSolrCoreSnapshots, TestSolrConfigHandler, OverseerTaskQueueTest, TestIBSimilarityFactory, TestIndexingPerformance, TestJsonFacetRefinement, SpatialRPTFieldTypeTest, FieldAnalysisRequestHandlerTest, SolrCoreCheckLockOnStartupTest, LukeRequestHandlerTest, TestStressInPlaceUpdates, TestManagedSchema, VersionInfoTest, SuggesterTSTTest, StressHdfsTest, SuggesterTest, CreateCollectionCleanupTest, DateFieldTest, TestCollectionAPIs, TestComponentsName, SolrCloudExampleTest, TestLuceneMatchVersion, ClusterStateTest, DistributedFacetPivotLargeTest, OverseerTest, DirectoryFactoryTest, LeaderElectionTest, TestAddFieldRealTimeGet, TestMergePolicyConfig, TestBinaryResponseWriter, BasicDistributedZkTest, BasicDistributedZk2Test, UnloadDistributedZkTest, LeaderElectionIntegrationTest, BasicZkTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, TestReplicationHandler]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestReplicationHandler -Dtests.seed=B35B4390D1EAEB95 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=no -Dtests.timezone=Atlantic/Jan_Mayen -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J2 | TestReplicationHandler (suite) <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: ObjectTracker found 4 object(s) that were not released!!! [SolrCore, InternalHttpClient, MockDirectoryWrapper, MockDirectoryWrapper]
   [junit4]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
   [junit4]    > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1025)
   [junit4]    > at org.apache.solr.core.SolrCore.reload(SolrCore.java:648)
   [junit4]    > at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1166)
   [junit4]    > at org.apache.solr.handler.IndexFetcher.lambda$reloadCore$0(IndexFetcher.java:877)
   [junit4]    > at java.lang.Thread.run(Thread.java:745)
   [junit4]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
   [junit4]    > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:298)
   [junit4]    > at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:218)
   [junit4]    > at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:252)
   [junit4]    > at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1209)
   [junit4]    > at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:695)
   [junit4]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:973)
   [junit4]    > at org.apache.solr.core.SolrCore.reload(SolrCore.java:648)
   [junit4]    > at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1166)
   [junit4]    > at org.apache.solr.handler.IndexFetcher.lambda$reloadCore$0(IndexFetcher.java:877)
   [junit4]    > at java.lang.Thread.run(Thread.java:745)
   [junit4]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
   [junit4]    > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]    > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
   [junit4]    > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:461)
   [junit4]    > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:326)
   [junit4]    > at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:415)
   [junit4]    > at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1179)
   [junit4]    > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]    > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
   [junit4]    > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
   [junit4]    > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
   [junit4]    > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]    > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    > at java.lang.Thread.run(Thread.java:745)
   [junit4]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
   [junit4]    > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]    > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
   [junit4]    > at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:359)
   [junit4]    > at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:721)
   [junit4]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:940)
   [junit4]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
   [junit4]    > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:975)
   [junit4]    > at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:595)
   [junit4]    > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
   [junit4]    > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]    > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]    > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]    > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    > at java.lang.Thread.run(Thread.java:745)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([B35B4390D1EAEB95]:0)
   [junit4]    > at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:302)
   [junit4]    > at java.lang.Thread.run(Thread.java:745)
   [junit4] Completed [337/702 (1!)] on J2 in 382.44s, 16 tests, 1 failure, 1 skipped <<< FAILURES!

[...truncated 44487 lines...]




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1281 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1281/

1 tests failed.
FAILED:  org.apache.solr.update.AutoCommitTest.testCommitWithin

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
        at __randomizedtesting.SeedInfo.seed([31601BA5F83E12A1:8BB274DD7B10FCB4]:0)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:896)
        at org.apache.solr.update.AutoCommitTest.testCommitWithin(AutoCommitTest.java:353)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
        xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>

        request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:889)
        ... 40 more




Build Log:
[...truncated 12739 lines...]
   [junit4] Suite: org.apache.solr.update.AutoCommitTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_31601BA5F83E12A1-001/init-core-data-001
   [junit4]   2> 4037578 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields
   [junit4]   2> 4037580 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 4037580 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 4037581 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 4037630 ERROR (Finalizer) [    ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=2105382249
   [junit4]   2> 4037631 WARN  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 4037633 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 4037669 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 4037938 WARN  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    ] o.a.s.s.IndexSchema [null] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 4037940 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 4038025 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@759752c7
   [junit4]   2> 4038030 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@759752c7
   [junit4]   2> 4038030 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@759752c7
   [junit4]   2> 4038049 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 4038107 WARN  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 4038109 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 4038138 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 4038351 WARN  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 4038380 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 4038387 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 4038387 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@759752c7
   [junit4]   2> 4038387 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 4038387 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_31601BA5F83E12A1-001/init-core-data-001/]
   [junit4]   2> 4038399 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1330821008, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 4038474 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 4038474 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 4038475 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 4038475 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 4038476 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=25, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.9326171875, noCFSRatio=0.0]
   [junit4]   2> 4038476 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@33d03d90[collection1] main]
   [junit4]   2> 4038477 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 4038487 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 4038516 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 4038516 INFO  (searcherExecutor-5232-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 4038516 INFO  (searcherExecutor-5232-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 4038516 INFO  (searcherExecutor-5232-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 4038516 INFO  (searcherExecutor-5232-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 4038517 INFO  (searcherExecutor-5232-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 4038517 INFO  (searcherExecutor-5232-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 4038517 INFO  (searcherExecutor-5232-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 4038517 INFO  (coreLoadExecutor-5231-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1564624124297347072
   [junit4]   2> 4038519 INFO  (searcherExecutor-5232-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 4038520 INFO  (searcherExecutor-5232-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 4038522 INFO  (searcherExecutor-5232-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@33d03d90[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4038522 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 4038524 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMaxDocs
   [junit4]   2> 4038525 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 4038526 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@32d03051[collection1] realtime]
   [junit4]   2> 4038527 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 2
   [junit4]   2> 4038527 ERROR (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    ] o.a.s.c.CorePropertiesLocator Couldn't load core descriptor from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrcore.properties:java.nio.file.NoSuchFileException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrcore.properties
   [junit4]   2> 4038528 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 4038608 WARN  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 4038610 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 4038642 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 4038872 WARN  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    ] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 4038875 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 4038898 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1
   [junit4]   2> 4038957 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@759752c7
   [junit4]   2> 4038957 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_31601BA5F83E12A1-001/init-core-data-001/]
   [junit4]   2> 4039097 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 4039097 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 4039099 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6df13c8f[collection1] main]
   [junit4]   2> 4039099 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 4039101 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 4039105 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 4039105 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1564624124913909760
   [junit4]   2> 4039105 INFO  (searcherExecutor-5237-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 4039108 INFO  (searcherExecutor-5237-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 4039108 INFO  (searcherExecutor-5237-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 4039108 INFO  (searcherExecutor-5237-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 4039109 INFO  (searcherExecutor-5237-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 4039109 INFO  (searcherExecutor-5237-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 4039109 INFO  (searcherExecutor-5237-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 4039109 INFO  (searcherExecutor-5237-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 4039109 INFO  (searcherExecutor-5237-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 4039112 INFO  (searcherExecutor-5237-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6df13c8f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4039113 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=19, maxMergedSegmentMB=99.90234375, floorSegmentMB=0.375, forceMergeDeletesPctAllowed=22.333029474871363, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 4039113 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 4039113 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@46f6cb74[collection1] main]
   [junit4]   2> 4039114 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@716beb55
   [junit4]   2> 4039117 INFO  (searcherExecutor-5237-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@46f6cb74[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4039117 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1902898005
   [junit4]   2> 4039128 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[0 (1564624124935929856)]} 0 2
   [junit4]   2> 4039129 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[1 (1564624124938027008)]} 0 3
   [junit4]   2> 4039129 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[2 (1564624124939075584)]} 0 3
   [junit4]   2> 4039154 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[3 (1564624124939075585)]} 0 28
   [junit4]   2> 4039160 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[4 (1564624124971581440)]} 0 34
   [junit4]   2> 4039188 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[5 (1564624124992552960)]} 0 62
   [junit4]   2> 4039188 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[6 (1564624125000941568)]} 0 63
   [junit4]   2> 4039189 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[7 (1564624125001990144)]} 0 63
   [junit4]   2> 4039189 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[8 (1564624125001990145)]} 0 63
   [junit4]   2> 4039189 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[9 (1564624125001990146)]} 0 63
   [junit4]   2> 4039189 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[10 (1564624125001990147)]} 0 64
   [junit4]   2> 4039190 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[11 (1564624125003038720)]} 0 64
   [junit4]   2> 4039190 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[12 (1564624125003038721)]} 0 64
   [junit4]   2> 4039190 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[13 (1564624125003038722)]} 0 65
   [junit4]   2> 4039191 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:1&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 4039204 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[14 (1564624125006184448)]} 0 78
   [junit4]   2> 4039204 INFO  (commitScheduler-5240-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
   [junit4]   2> 4039209 INFO  (commitScheduler-5240-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@15be8d8b[collection1] main]
   [junit4]   2> 4039210 INFO  (commitScheduler-5240-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 4039212 INFO  (searcherExecutor-5237-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@15be8d8b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):c15)))}
   [junit4]   2> 4039455 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[15 (1564624125279862784)]} 0 329
   [junit4]   2> 4039455 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:14&qt=standard&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
   [junit4]   2> 4039456 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:15&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 4039456 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testMaxDocs
   [junit4]   2> 4039459 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Starting testCommitWithin
   [junit4]   2> 4039460 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 4039460 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@13a1f975[collection1] realtime]
   [junit4]   2> 4039482 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 22
   [junit4]   2> 4039482 ERROR (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.CorePropertiesLocator Couldn't load core descriptor from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrcore.properties:java.nio.file.NoSuchFileException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrcore.properties
   [junit4]   2> 4039483 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 4039532 WARN  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 4039533 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 4039557 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 4039709 WARN  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 4039712 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 4039718 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1
   [junit4]   2> 4039738 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@759752c7
   [junit4]   2> 4039738 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_31601BA5F83E12A1-001/init-core-data-001/]
   [junit4]   2> 4040019 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 4040019 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 4040021 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@66f5275a[collection1] main]
   [junit4]   2> 4040022 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 4040022 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 4040026 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 4040028 INFO  (searcherExecutor-5241-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 4040029 INFO  (searcherExecutor-5241-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 4040029 INFO  (searcherExecutor-5241-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 4040029 INFO  (searcherExecutor-5241-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 4040029 INFO  (searcherExecutor-5241-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 4040030 INFO  (searcherExecutor-5241-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 4040030 INFO  (searcherExecutor-5241-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 4040030 INFO  (searcherExecutor-5241-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 4040030 INFO  (searcherExecutor-5241-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 4040033 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=34, maxMergedSegmentMB=17.01171875, floorSegmentMB=0.8037109375, forceMergeDeletesPctAllowed=28.2563834767927, segmentsPerTier=4.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.448952022933381
   [junit4]   2> 4040034 INFO  (searcherExecutor-5241-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@66f5275a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4040034 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 4040034 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@236c4abb[collection1] main]
   [junit4]   2> 4040035 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1523ae04
   [junit4]   2> 4040035 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=354659844
   [junit4]   2> 4040040 INFO  (searcherExecutor-5241-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@236c4abb[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4040041 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[529 (1564624125892231168)]} 0 2
   [junit4]   2> 4040041 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 4040042 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[530 (1564624125896425472)]} 0 3
   [junit4]   2> 4040042 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:530&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 4040043 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[529 (-1564624125897474048)]} 0 4
   [junit4]   2> 4040043 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 4040043 ERROR (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2>
   [junit4]   2> request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
   [junit4]   2> 4040056 ERROR (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: q=id:529&qt=standard&start=0&rows=20&version=2.2:java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2>
   [junit4]   2> request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
   [junit4]   2> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:889)
   [junit4]   2> at org.apache.solr.update.AutoCommitTest.testCommitWithin(AutoCommitTest.java:353)
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>
   [junit4]   2> 4040056 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testCommitWithin
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=AutoCommitTest -Dtests.method=testCommitWithin -Dtests.seed=31601BA5F83E12A1 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=hi -Dtests.timezone=Asia/Tbilisi -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.61s J2 | AutoCommitTest.testCommitWithin <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: Exception during query
   [junit4]    > at __randomizedtesting.SeedInfo.seed([31601BA5F83E12A1:8BB274DD7B10FCB4]:0)
   [junit4]    > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:896)
   [junit4]    > at org.apache.solr.update.AutoCommitTest.testCommitWithin(AutoCommitTest.java:353)
   [junit4]    > at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]    > xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]    > <response>
   [junit4]    > <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]    > </response>
   [junit4]    > request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
   [junit4]    > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:889)
   [junit4]    > ... 40 more
   [junit4]   2> 4040078 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Starting testMaxTime
   [junit4]   2> 4040078 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 4040078 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@58eaa436[collection1] realtime]
   [junit4]   2> 4040079 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 0
   [junit4]   2> 4040079 ERROR (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.CorePropertiesLocator Couldn't load core descriptor from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrcore.properties:java.nio.file.NoSuchFileException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrcore.properties
   [junit4]   2> 4040080 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 4040133 WARN  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 4040135 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 4040159 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 4040424 WARN  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 4040427 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 4040437 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1
   [junit4]   2> 4040462 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@759752c7
   [junit4]   2> 4040462 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_31601BA5F83E12A1-001/init-core-data-001/]
   [junit4]   2> 4040554 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 4040554 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 4040556 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6fa3ce9e[collection1] main]
   [junit4]   2> 4040557 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 4040558 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 4040566 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 4040569 INFO  (searcherExecutor-5245-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 4040569 INFO  (searcherExecutor-5245-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 4040569 INFO  (searcherExecutor-5245-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 4040569 INFO  (searcherExecutor-5245-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 4040569 INFO  (searcherExecutor-5245-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 4040569 INFO  (searcherExecutor-5245-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 4040570 INFO  (searcherExecutor-5245-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 4040570 INFO  (searcherExecutor-5245-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 4040570 INFO  (searcherExecutor-5245-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 4040573 INFO  (searcherExecutor-5245-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6fa3ce9e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4040574 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=598179592, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 4040574 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 4040575 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3bb72b6e[collection1] main]
   [junit4]   2> 4040575 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@498f085
   [junit4]   2> 4040576 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=77131909
   [junit4]   2> 4040583 INFO  (searcherExecutor-5245-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3bb72b6e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4040586 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[529 (1564624126465802240)]} 0 0
   [junit4]   2> 4040586 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 4042086 INFO  (commitScheduler-5248-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
   [junit4]   2> 4042089 INFO  (commitScheduler-5248-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5f2b7f90[collection1] main]
   [junit4]   2> 4042090 INFO  (commitScheduler-5248-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 4042091 INFO  (searcherExecutor-5245-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5f2b7f90[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_3(7.0.0):c1)))}
   [junit4]   2> 4042344 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[530 (1564624128302907392)]} 0 1759
   [junit4]   2> 4042344 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
   [junit4]   2> 4042345 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:530&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 4042346 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{delete=[529 (-1564624128312344576)]} 0 0
   [junit4]   2> 4042346 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
   [junit4]   2> 4043844 INFO  (commitScheduler-5248-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
   [junit4]   2> 4043850 INFO  (commitScheduler-5248-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@780a62fc[collection1] main]
   [junit4]   2> 4043850 INFO  (commitScheduler-5248-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 4043852 INFO  (searcherExecutor-5245-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@780a62fc[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_4(7.0.0):c1)))}
   [junit4]   2> 4044098 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[550 (1564624130148401152)]} 0 3513
   [junit4]   2> 4044098 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 4044099 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1564624130150498304)]} 0 3514
   [junit4]   2> 4044099 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1564624130150498305)]} 0 3514
   [junit4]   2> 4044099 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1564624130150498306)]} 0 3514
   [junit4]   2> 4044100 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1564624130150498307)]} 0 3514
   [junit4]   2> 4044100 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1564624130151546880)]} 0 3514
   [junit4]   2> 4044100 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1564624130151546881)]} 0 3515
   [junit4]   2> 4044108 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1564624130151546882)]} 0 3522
   [junit4]   2> 4044108 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1564624130159935488)]} 0 3523
   [junit4]   2> 4044108 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1564624130159935489)]} 0 3523
   [junit4]   2> 4044109 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1564624130159935490)]} 0 3523
   [junit4]   2> 4044109 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:500&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 4045598 INFO  (commitScheduler-5248-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
   [junit4]   2> 4045601 INFO  (commitScheduler-5248-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@35c144be[collection1] main]
   [junit4]   2> 4045602 INFO  (commitScheduler-5248-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 4045603 INFO  (searcherExecutor-5245-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@35c144be[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_4(7.0.0):c1) Uninverting(_5(7.0.0):c11/9:delGen=1)))}
   [junit4]   2> 4045614 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[531 (1564624131734896640)]} 0 5029
   [junit4]   2> 4045614 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:500&qt=standard&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
   [junit4]   2> 4045615 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[31601BA5F83E12A1]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testMaxTime
   [junit4]   2> 4045615 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 4045615 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    x:collection1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2113695735
   [junit4]   2> 4045615 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 4045617 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 4045624 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 4045624 INFO  (SUITE-AutoCommitTest-seed#[31601BA5F83E12A1]-worker) [    x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 4045630 INFO  (coreCloseExecutor-5249-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@2ba2b3a0
   [junit4]   2> 4045630 INFO  (coreCloseExecutor-5249-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter close.
   [junit4]   2> 4045630 INFO  (coreCloseExecutor-5249-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@26e5e60d commitCommandVersion:0
   [junit4]   2> 4045652 INFO  (coreCloseExecutor-5249-thread-1-processing-x:collection1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=732083104
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_31601BA5F83E12A1-001
   [junit4]   2> Apr 14, 2017 3:50:49 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=32355, maxDocsPerChunk=2, blockSize=543), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=32355, blockSize=543)), sim=RandomSimilarity(queryNorm=false): {}, locale=hi, timezone=Asia/Tbilisi
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=251540024,total=513802240
   [junit4]   2> NOTE: All tests run in this JVM: [TestDistribDocBasedVersion, SolrMetricReporterTest, SynonymTokenizerTest, DirectoryFactoryTest, ClassificationUpdateProcessorFactoryTest, BadIndexSchemaTest, SpatialRPTFieldTypeTest, BlockDirectoryTest, TestSolrFieldCacheBean, ResponseHeaderTest, UnloadDistributedZkTest, TestTestInjection, TestTrackingShardHandlerFactory, ShowFileRequestHandlerTest, SolrIndexConfigTest, TestRequestStatusCollectionAPI, EchoParamsTest, TestReqParamsAPI, HdfsLockFactoryTest, HdfsRestartWhileUpdatingTest, SuggesterTest, TestStressInPlaceUpdates, StatsComponentTest, TestFreeTextSuggestions, TestSolrQueryResponse, ParsingFieldUpdateProcessorsTest, FieldMutatingUpdateProcessorTest, TestCollationField, OutputWriterTest, MigrateRouteKeyTest, CollectionTooManyReplicasTest, TestSolrQueryParser, TestTolerantUpdateProcessorRandomCloud, TestCloudPseudoReturnFields, RulesTest, DataDrivenBlockJoinTest, TestManagedResourceStorage, SimpleFacetsTest, HdfsTlogReplayBufferedWhileIndexingTest, CdcrReplicationDistributedZkTest, TestRuleBasedAuthorizationPlugin, TestLeaderInitiatedRecoveryThread, TestConfigSetsAPIZkFailure, TestSystemCollAutoCreate, TestZkChroot, RemoteQueryErrorTest, TestFilteredDocIdSet, DeleteShardTest, TestSolrQueryParserResource, TestDistributedStatsComponentCardinality, TestAnalyzeInfixSuggestions, StatelessScriptUpdateProcessorFactoryTest, TestRebalanceLeaders, TestClassicSimilarityFactory, TestFieldCacheReopen, HttpPartitionTest, TestExtendedDismaxParser, LeaderElectionTest, TestIndexSearcher, TestAddFieldRealTimeGet, CdcrVersionReplicationTest, BasicDistributedZk2Test, NoCacheHeaderTest, CurrencyFieldXmlFileTest, LeaderFailureAfterFreshStartTest, TestMiniSolrCloudCluster, FieldAnalysisRequestHandlerTest, WrapperMergePolicyFactoryTest, TestStressReorder, FullSolrCloudDistribCmdsTest, TestSolrConfigHandler, CachingDirectoryFactoryTest, PrimitiveFieldTypeTest, TestOnReconnectListenerSupport, HighlighterConfigTest, TestFieldResource, MergeStrategyTest, RestartWhileUpdatingTest, SolrPluginUtilsTest, TestDocBasedVersionConstraints, TestBinaryField, TestComponentsName, TestPhraseSuggestions, TestImplicitCoreProperties, DeleteLastCustomShardedReplicaTest, ChangedSchemaMergeTest, HdfsChaosMonkeyNothingIsSafeTest, TestConfigReload, TestClassNameShortening, MetricsHandlerTest, DistributedVersionInfoTest, ZkStateWriterTest, DebugComponentTest, TestIndexingPerformance, TestSmileRequest, TestReloadDeadlock, RecoveryZkTest, TestUnifiedSolrHighlighter, StressHdfsTest, SaslZkACLProviderTest, DistributedDebugComponentTest, TestConfig, XmlUpdateRequestHandlerTest, UUIDUpdateProcessorFallbackTest, HdfsNNFailoverTest, InfixSuggestersTest, TestSQLHandlerNonCloud, TestDocSet, ClusterStateTest, TestSchemaVersionResource, SolrCloudReportersTest, OutOfBoxZkACLAndCredentialsProvidersTest, AnalyticsQueryTest, BasicDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, ClusterStateUpdateTest, ZkSolrClientTest, ZkCLITest, AutoCommitTest]
   [junit4] Completed [349/702 (1!)] on J2 in 8.35s, 3 tests, 1 error <<< FAILURES!

[...truncated 44415 lines...]




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1282 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1282/

1 tests failed.
FAILED:  org.apache.solr.update.AutoCommitTest.testMaxTime

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
        at __randomizedtesting.SeedInfo.seed([D532F2C287D19F90:4FC68F20194B03AC]:0)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:896)
        at org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:270)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
        xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>

        request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:889)
        ... 40 more




Build Log:
[...truncated 12146 lines...]
   [junit4] Suite: org.apache.solr.update.AutoCommitTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_D532F2C287D19F90-001/init-core-data-001
   [junit4]   2> 1868754 WARN  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=15 numCloses=15
   [junit4]   2> 1868754 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields
   [junit4]   2> 1868756 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1868756 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 1868757 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1868904 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1869019 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 1869449 WARN  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    ] o.a.s.s.IndexSchema [null] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 1869487 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1869600 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3a7dcf6c
   [junit4]   2> 1869613 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3a7dcf6c
   [junit4]   2> 1869613 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3a7dcf6c
   [junit4]   2> 1869616 INFO  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1869743 INFO  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1869766 INFO  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1869929 WARN  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 1869942 INFO  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1869949 INFO  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 1869949 INFO  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3a7dcf6c
   [junit4]   2> 1869950 INFO  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1869950 INFO  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_D532F2C287D19F90-001/init-core-data-001/]
   [junit4]   2> 1869952 INFO  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=32.4765625, floorSegmentMB=0.2431640625, forceMergeDeletesPctAllowed=12.345212809736296, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6125864285292415
   [junit4]   2> 1870048 INFO  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1870048 INFO  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1870049 INFO  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=618428531, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1870049 INFO  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1136d3fd[collection1] main]
   [junit4]   2> 1870053 INFO  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1870061 INFO  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1870101 INFO  (coreLoadExecutor-2970-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1870101 INFO  (searcherExecutor-2971-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1870103 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 1870106 INFO  (searcherExecutor-2971-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 1870106 INFO  (searcherExecutor-2971-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1870106 INFO  (searcherExecutor-2971-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1870108 INFO  (searcherExecutor-2971-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1870108 INFO  (searcherExecutor-2971-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 1870111 INFO  (searcherExecutor-2971-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 1870115 INFO  (searcherExecutor-2971-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 1870116 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMaxDocs
   [junit4]   2> 1870116 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 1870116 INFO  (searcherExecutor-2971-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 1870118 INFO  (searcherExecutor-2971-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1136d3fd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1870118 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:*} 0 2
   [junit4]   2> 1870119 ERROR (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    ] o.a.s.c.CorePropertiesLocator Couldn't load core descriptor from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrcore.properties:java.nio.file.NoSuchFileException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrcore.properties
   [junit4]   2> 1870119 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1870167 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1870281 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1870553 WARN  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    ] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 1870586 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1870621 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1
   [junit4]   2> 1870838 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3a7dcf6c
   [junit4]   2> 1870839 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_D532F2C287D19F90-001/init-core-data-001/]
   [junit4]   2> 1870931 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1870931 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1870940 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7e2420e1[collection1] main]
   [junit4]   2> 1870950 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1870954 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1870967 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1870967 INFO  (searcherExecutor-2975-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1870967 INFO  (searcherExecutor-2975-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 1870967 INFO  (searcherExecutor-2975-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1870967 INFO  (searcherExecutor-2975-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1870967 INFO  (searcherExecutor-2975-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1870968 INFO  (searcherExecutor-2975-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 1870968 INFO  (searcherExecutor-2975-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 1870969 INFO  (searcherExecutor-2975-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 1870969 INFO  (searcherExecutor-2975-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 1870972 INFO  (searcherExecutor-2975-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7e2420e1[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1870974 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@42380585
   [junit4]   2> 1870974 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1870975 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@244be184[collection1] main]
   [junit4]   2> 1870978 INFO  (searcherExecutor-2975-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@244be184[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1870978 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@371578f6
   [junit4]   2> 1870978 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=924154102
   [junit4]   2> 1871016 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[0]} 0 1
   [junit4]   2> 1871016 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[1]} 0 1
   [junit4]   2> 1871016 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[2]} 0 2
   [junit4]   2> 1871018 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[3]} 0 3
   [junit4]   2> 1871018 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[4]} 0 3
   [junit4]   2> 1871018 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[5]} 0 4
   [junit4]   2> 1871019 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[6]} 0 4
   [junit4]   2> 1871019 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[7]} 0 4
   [junit4]   2> 1871019 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[8]} 0 5
   [junit4]   2> 1871020 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[9]} 0 5
   [junit4]   2> 1871020 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[10]} 0 5
   [junit4]   2> 1871020 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[11]} 0 6
   [junit4]   2> 1871020 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[12]} 0 6
   [junit4]   2> 1871021 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[13]} 0 6
   [junit4]   2> 1871021 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:1&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1871024 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[14]} 0 10
   [junit4]   2> 1871032 INFO  (commitScheduler-2978-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
   [junit4]   2> 1871039 INFO  (commitScheduler-2978-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3e97f58f[collection1] main]
   [junit4]   2> 1871040 INFO  (commitScheduler-2978-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1871042 INFO  (searcherExecutor-2975-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3e97f58f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C15)))}
   [junit4]   2> 1871275 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[15]} 0 261
   [junit4]   2> 1871276 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:14&qt=standard&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
   [junit4]   2> 1871277 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:15&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1871277 INFO  (TEST-AutoCommitTest.testMaxDocs-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testMaxDocs
   [junit4]   2> 1871280 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Starting testMaxTime
   [junit4]   2> 1871280 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 1871281 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:*} 0 0
   [junit4]   2> 1871281 ERROR (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.CorePropertiesLocator Couldn't load core descriptor from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrcore.properties:java.nio.file.NoSuchFileException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrcore.properties
   [junit4]   2> 1871282 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1871330 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1871359 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1871512 WARN  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 1871514 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1871522 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1
   [junit4]   2> 1871539 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3a7dcf6c
   [junit4]   2> 1871539 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_D532F2C287D19F90-001/init-core-data-001/]
   [junit4]   2> 1871612 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1871612 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1871615 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5b04cbe5[collection1] main]
   [junit4]   2> 1871616 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1871617 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1871628 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1871629 INFO  (searcherExecutor-2979-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1871630 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=35, maxMergedSegmentMB=1.6904296875, floorSegmentMB=0.853515625, forceMergeDeletesPctAllowed=1.3800556768668448, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1871630 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1871631 INFO  (searcherExecutor-2979-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 1871631 INFO  (searcherExecutor-2979-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1871631 INFO  (searcherExecutor-2979-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1871631 INFO  (searcherExecutor-2979-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1871631 INFO  (searcherExecutor-2979-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 1871632 INFO  (searcherExecutor-2979-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 1871632 INFO  (searcherExecutor-2979-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 1871632 INFO  (searcherExecutor-2979-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 1871635 INFO  (searcherExecutor-2979-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5b04cbe5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1871635 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@67e98c71[collection1] main]
   [junit4]   2> 1871636 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@3e904111
   [junit4]   2> 1871638 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1049641233
   [junit4]   2> 1871719 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[529]} 0 75
   [junit4]   2> 1871721 INFO  (searcherExecutor-2979-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@67e98c71[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1871722 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=2
   [junit4]   2> 1871723 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[530]} 0 80
   [junit4]   2> 1871723 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1871723 ERROR (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2>
   [junit4]   2> request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
   [junit4]   2> 1871724 ERROR (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: q=id:529&qt=standard&start=0&rows=20&version=2.2:java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2>
   [junit4]   2> request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
   [junit4]   2> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:889)
   [junit4]   2> at org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:270)
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>
   [junit4]   2> 1871724 INFO  (TEST-AutoCommitTest.testMaxTime-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testMaxTime
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=AutoCommitTest -Dtests.method=testMaxTime -Dtests.seed=D532F2C287D19F90 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=bg -Dtests.timezone=Europe/Vienna -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.45s J2 | AutoCommitTest.testMaxTime <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: Exception during query
   [junit4]    > at __randomizedtesting.SeedInfo.seed([D532F2C287D19F90:4FC68F20194B03AC]:0)
   [junit4]    > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:896)
   [junit4]    > at org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:270)
   [junit4]    > at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]    > xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]    > <response>
   [junit4]    > <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]    > </response>
   [junit4]    > request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
   [junit4]    > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:889)
   [junit4]    > ... 40 more
   [junit4]   2> 1871742 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Starting testCommitWithin
   [junit4]   2> 1871743 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 1871743 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:*} 0 0
   [junit4]   2> 1871743 ERROR (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.CorePropertiesLocator Couldn't load core descriptor from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrcore.properties:java.nio.file.NoSuchFileException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrcore.properties
   [junit4]   2> 1871744 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1871794 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1871829 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1871966 WARN  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 1871969 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1871976 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1
   [junit4]   2> 1871999 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3a7dcf6c
   [junit4]   2> 1871999 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_D532F2C287D19F90-001/init-core-data-001/]
   [junit4]   2> 1872126 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1872126 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1872127 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@66ced051[collection1] main]
   [junit4]   2> 1872128 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1872129 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1872132 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1872133 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=34, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5961098941651091]
   [junit4]   2> 1872133 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1872133 INFO  (searcherExecutor-2983-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1872134 INFO  (searcherExecutor-2983-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 1872134 INFO  (searcherExecutor-2983-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1872134 INFO  (searcherExecutor-2983-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1872135 INFO  (searcherExecutor-2983-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1872135 INFO  (searcherExecutor-2983-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 1872135 INFO  (searcherExecutor-2983-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 1872135 INFO  (searcherExecutor-2983-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 1872136 INFO  (searcherExecutor-2983-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 1872138 INFO  (searcherExecutor-2983-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@66ced051[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1872139 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7dc9760b[collection1] main]
   [junit4]   2> 1872139 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@6e9c842d
   [junit4]   2> 1872140 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1855751213
   [junit4]   2> 1872160 INFO  (searcherExecutor-2983-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7dc9760b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1872180 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[529]} 0 18
   [junit4]   2> 1872181 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1876163 INFO  (commitScheduler-2986-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
   [junit4]   2> 1876172 INFO  (commitScheduler-2986-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1cabf6ea[collection1] main]
   [junit4]   2> 1876172 INFO  (commitScheduler-2986-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1876174 INFO  (searcherExecutor-2983-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1cabf6ea[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_3(7.0.0):C1)))}
   [junit4]   2> 1876274 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[530]} 0 4112
   [junit4]   2> 1876274 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:530&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1876276 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[529]} 0 4114
   [junit4]   2> 1876276 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
   [junit4]   2> 1878276 INFO  (commitScheduler-2986-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
   [junit4]   2> 1878279 INFO  (commitScheduler-2986-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@27b8fa08[collection1] main]
   [junit4]   2> 1878279 INFO  (commitScheduler-2986-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1878291 INFO  (searcherExecutor-2983-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@27b8fa08[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_4(7.0.0):C1)))}
   [junit4]   2> 1878292 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1878293 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:530&qt=standard&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
   [junit4]   2> 1878294 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500]} 0 6132
   [junit4]   2> 1878294 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500]} 0 6132
   [junit4]   2> 1878294 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500]} 0 6132
   [junit4]   2> 1878294 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500]} 0 6132
   [junit4]   2> 1878295 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500]} 0 6133
   [junit4]   2> 1878295 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500]} 0 6133
   [junit4]   2> 1878295 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500]} 0 6133
   [junit4]   2> 1878295 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500]} 0 6133
   [junit4]   2> 1878295 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500]} 0 6133
   [junit4]   2> 1878296 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500]} 0 6134
   [junit4]   2> 1878296 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:500&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1878296 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530]} 0 6134
   [junit4]   2> 1878296 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530]} 0 6134
   [junit4]   2> 1878297 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530]} 0 6135
   [junit4]   2> 1878297 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530]} 0 6135
   [junit4]   2> 1878297 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530]} 0 6135
   [junit4]   2> 1878297 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530]} 0 6135
   [junit4]   2> 1878297 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530]} 0 6135
   [junit4]   2> 1878297 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530]} 0 6135
   [junit4]   2> 1878298 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530]} 0 6136
   [junit4]   2> 1878298 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530]} 0 6136
   [junit4]   2> 1878298 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:530&qt=standard&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
   [junit4]   2> 1879297 INFO  (commitScheduler-2986-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
   [junit4]   2> 1879299 INFO  (commitScheduler-2986-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@59aa01ad[collection1] main]
   [junit4]   2> 1879300 INFO  (commitScheduler-2986-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1879302 INFO  (searcherExecutor-2983-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@59aa01ad[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_5(7.0.0):C10/9:delGen=1)))}
   [junit4]   2> 1879302 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:500&qt=standard&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
   [junit4]   2> 1879303 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:530&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1879303 INFO  (TEST-AutoCommitTest.testCommitWithin-seed#[D532F2C287D19F90]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testCommitWithin
   [junit4]   2> 1879304 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 1879304 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    x:collection1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1556281506
   [junit4]   2> 1879304 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1879305 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1879308 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1879309 INFO  (SUITE-AutoCommitTest-seed#[D532F2C287D19F90]-worker) [    x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1879309 INFO  (coreCloseExecutor-2987-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@53339bb8
   [junit4]   2> 1879312 INFO  (coreCloseExecutor-2987-thread-1-processing-x:collection1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1395891128
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_D532F2C287D19F90-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {field_t=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=PostingsFormat(name=Asserting), subject=PostingsFormat(name=Asserting), id=FST50, text=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Asserting), intDefault=DocValuesFormat(name=Memory), range_facet_i_dv=DocValuesFormat(name=Memory), intDvoDefault=DocValuesFormat(name=Lucene70), range_facet_l=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=869, maxMBSortInHeap=5.567957916056452, sim=RandomSimilarity(queryNorm=true): {}, locale=bg, timezone=Europe/Vienna
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=180879512,total=471859200
   [junit4]   2> NOTE: All tests run in this JVM: [DeleteShardTest, SuggesterTSTTest, DirectUpdateHandlerTest, TestCSVResponseWriter, ShowFileRequestHandlerTest, DirectUpdateHandlerOptimizeTest, DirectoryFactoryTest, TestTrackingShardHandlerFactory, TestRuleBasedAuthorizationPlugin, TestLMDirichletSimilarityFactory, TestInPlaceUpdatesDistrib, TestManagedSchema, TestSchemaVersionResource, TestMergePolicyConfig, TestCollationField, TestDistributedStatsComponentCardinality, CloneFieldUpdateProcessorFactoryTest, TestNumericTerms32, TestNRTOpen, TestTolerantSearch, ExternalFileFieldSortTest, TestBinaryResponseWriter, SuggesterWFSTTest, BadIndexSchemaTest, TestStressInPlaceUpdates, TestCoreDiscovery, HighlighterConfigTest, TestPHPSerializedResponseWriter, TestRequestStatusCollectionAPI, TestFreeTextSuggestions, FieldMutatingUpdateProcessorTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestMiniSolrCloudCluster, OverseerTest, TestCloudPseudoReturnFields, BinaryUpdateRequestHandlerTest, HdfsNNFailoverTest, TestFieldCacheVsDocValues, TestBinaryField, InfixSuggestersTest, BlockJoinFacetDistribTest, MBeansHandlerTest, TestDistribIDF, SolrCoreCheckLockOnStartupTest, DataDrivenBlockJoinTest, TestDocSet, UnloadDistributedZkTest, TestClassNameShortening, UpdateParamsTest, TestCollectionAPIs, TestTolerantUpdateProcessorRandomCloud, HardAutoCommitTest, TestEmbeddedSolrServerSchemaAPI, TestSchemaNameResource, SolrTestCaseJ4Test, RestartWhileUpdatingTest, TestSmileRequest, DistributedFacetPivotLargeTest, TestComplexPhraseQParserPlugin, BigEndianAscendingWordSerializerTest, TestFieldCacheReopen, AddBlockUpdateTest, AutoCommitTest]
   [junit4] Completed [154/702 (1!)] on J2 in 10.64s, 3 tests, 1 error <<< FAILURES!

[...truncated 45090 lines...]




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1283 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1283/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CdcrBootstrapTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [RawDirectoryWrapper] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.RawDirectoryWrapper  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)  at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:461)  at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:326)  at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:415)  at org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:757)  at org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:712)  at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:745)  

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [RawDirectoryWrapper]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.RawDirectoryWrapper
        at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
        at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:461)
        at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:326)
        at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:415)
        at org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:757)
        at org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:712)
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)


        at __randomizedtesting.SeedInfo.seed([42E99CA6451700F9]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:302)
        at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv

Error Message:
java.lang.RuntimeException: Error from server at http://127.0.0.1:58070/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:32886/solr/test_col_shard5_replica2: Server Error    request: http://127.0.0.1:32886/solr/test_col_shard5_replica2/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A58070%2Fsolr%2Ftest_col_shard5_replica1%2F&wt=javabin&version=2 Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:58070/solr/test_col_shard5_replica1/ update: org.apache.solr.client.solrj.request.UpdateRequest@6210d956

Stack Trace:
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error from server at http://127.0.0.1:58070/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:32886/solr/test_col_shard5_replica2: Server Error



request: http://127.0.0.1:32886/solr/test_col_shard5_replica2/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A58070%2Fsolr%2Ftest_col_shard5_replica1%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:58070/solr/test_col_shard5_replica1/ update: org.apache.solr.client.solrj.request.UpdateRequest@6210d956
        at __randomizedtesting.SeedInfo.seed([42E99CA6451700F9:74FDFEE0CF4A3AE8]:0)
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:281)
        at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:193)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Error from server at http://127.0.0.1:58070/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:32886/solr/test_col_shard5_replica2: Server Error



request: http://127.0.0.1:32886/solr/test_col_shard5_replica2/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A58070%2Fsolr%2Ftest_col_shard5_replica1%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:58070/solr/test_col_shard5_replica1/ update: org.apache.solr.client.solrj.request.UpdateRequest@6210d956
        at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:409)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        ... 1 more
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58070/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:32886/solr/test_col_shard5_replica2: Server Error



request: http://127.0.0.1:32886/solr/test_col_shard5_replica2/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A58070%2Fsolr%2Ftest_col_shard5_replica1%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:58070/solr/test_col_shard5_replica1/ update: org.apache.solr.client.solrj.request.UpdateRequest@6210d956
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
        at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.doRandomAtomicUpdate(TestStressCloudBlindAtomicUpdates.java:368)
        at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:397)
        ... 6 more


FAILED:  org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest.test

Error Message:
Mismatch in counts between replicas

Stack Trace:
java.lang.AssertionError: Mismatch in counts between replicas
        at __randomizedtesting.SeedInfo.seed([42E99CA6451700F9:CABDA37CEBEB6D01]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.RecoveryZkTest.assertShardConsistency(RecoveryZkTest.java:143)
        at org.apache.solr.cloud.RecoveryZkTest.test(RecoveryZkTest.java:126)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1492484053196,generation=2,filelist=[_0.cfe, _0.cfs, _0.si, _1.cfe, _1.cfs, _1.si, _2.cfe, _2.cfs, _2.si, _3.cfe, _3.cfs, _3.si, _4.cfe, _4.cfs, _4.si, _5.cfe, _5.cfs, _5.si, segments_2]}]> but was:<[{indexVersion=1492484053196,generation=2,filelist=[_0.cfe, _0.cfs, _0.si, _1.cfe, _1.cfs, _1.si, _2.cfe, _2.cfs, _2.si, _3.cfe, _3.cfs, _3.si, _4.cfe, _4.cfs, _4.si, _5.cfe, _5.cfs, _5.si, segments_2]}, {indexVersion=1492484053196,generation=3,filelist=[_6.fdt, _6.fdx, _6.fnm, _6.nvd, _6.nvm, _6.si, _6_Lucene50_0.doc, _6_Lucene50_0.tim, _6_Lucene50_0.tip, segments_3]}]>

Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1492484053196,generation=2,filelist=[_0.cfe, _0.cfs, _0.si, _1.cfe, _1.cfs, _1.si, _2.cfe, _2.cfs, _2.si, _3.cfe, _3.cfs, _3.si, _4.cfe, _4.cfs, _4.si, _5.cfe, _5.cfs, _5.si, segments_2]}]> but was:<[{indexVersion=1492484053196,generation=2,filelist=[_0.cfe, _0.cfs, _0.si, _1.cfe, _1.cfs, _1.si, _2.cfe, _2.cfs, _2.si, _3.cfe, _3.cfs, _3.si, _4.cfe, _4.cfs, _4.si, _5.cfe, _5.cfs, _5.si, segments_2]}, {indexVersion=1492484053196,generation=3,filelist=[_6.fdt, _6.fdx, _6.fnm, _6.nvd, _6.nvm, _6.si, _6_Lucene50_0.doc, _6_Lucene50_0.tim, _6_Lucene50_0.tip, segments_3]}]>
        at __randomizedtesting.SeedInfo.seed([42E99CA6451700F9:673E8796355F0EFA]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:147)
        at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1279)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11679 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_42E99CA6451700F9-001/init-core-data-001
   [junit4]   2> 51182 WARN  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[42E99CA6451700F9]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 51183 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[42E99CA6451700F9]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields
   [junit4]   2> 51186 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[42E99CA6451700F9]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=SSL overhead seems to cause OutOfMemory when stress testing)
   [junit4]   2> 51191 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[42E99CA6451700F9]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 10 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_42E99CA6451700F9-001/tempDir-001
   [junit4]   2> 51192 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[42E99CA6451700F9]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 51298 INFO  (Thread-11) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 51298 INFO  (Thread-11) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 51340 INFO  (SUITE-TestStressCloudBlindAtomicUpdates-seed#[42E99CA6451700F9]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:46794
   [junit4]   2> 52380 INFO  (jetty-launcher-26-thread-9) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 52388 INFO  (jetty-launcher-26-thread-10) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 52396 INFO  (jetty-launcher-26-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 52614 INFO  (jetty-launcher-26-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 52615 INFO  (jetty-launcher-26-thread-3) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 52616 INFO  (jetty-launcher-26-thread-6) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 52616 INFO  (jetty-launcher-26-thread-5) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 52617 INFO  (jetty-launcher-26-thread-4) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 52618 INFO  (jetty-launcher-26-thread-8) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 52621 INFO  (jetty-launcher-26-thread-7) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 53146 INFO  (jetty-launcher-26-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@649367a4{/solr,null,AVAILABLE}
   [junit4]   2> 53146 INFO  (jetty-launcher-26-thread-10) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@70054823{/solr,null,AVAILABLE}
   [junit4]   2> 53146 INFO  (jetty-launcher-26-thread-8) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3c2eb478{/solr,null,AVAILABLE}
   [junit4]   2> 53209 INFO  (jetty-launcher-26-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@628a882d{/solr,null,AVAILABLE}
   [junit4]   2> 53150 INFO  (jetty-launcher-26-thread-9) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@381816e5{/solr,null,AVAILABLE}
   [junit4]   2> 53232 INFO  (jetty-launcher-26-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6fde00db{/solr,null,AVAILABLE}
   [junit4]   2> 53271 INFO  (jetty-launcher-26-thread-7) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@ddd897a{/solr,null,AVAILABLE}
   [junit4]   2> 53301 INFO  (jetty-launcher-26-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@68da7da1{/solr,null,AVAILABLE}
   [junit4]   2> 53317 INFO  (jetty-launcher-26-thread-5) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6cf54549{/solr,null,AVAILABLE}
   [junit4]   2> 53329 INFO  (jetty-launcher-26-thread-6) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@d105a5e{/solr,null,AVAILABLE}
   [junit4]   2> 53365 INFO  (jetty-launcher-26-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1eacaeb2{HTTP/1.1,[http/1.1]}{127.0.0.1:59203}
   [junit4]   2> 53365 INFO  (jetty-launcher-26-thread-3) [    ] o.e.j.s.Server Started @61363ms
   [junit4]   2> 53365 INFO  (jetty-launcher-26-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59203}
   [junit4]   2> 53368 INFO  (jetty-launcher-26-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@21fd9d5{HTTP/1.1,[http/1.1]}{127.0.0.1:59189}
   [junit4]   2> 53371 INFO  (jetty-launcher-26-thread-2) [    ] o.e.j.s.Server Started @61370ms
   [junit4]   2> 53371 INFO  (jetty-launcher-26-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59189}
   [junit4]   2> 53368 INFO  (jetty-launcher-26-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7f271c9f{HTTP/1.1,[http/1.1]}{127.0.0.1:58070}
   [junit4]   2> 53372 INFO  (jetty-launcher-26-thread-1) [    ] o.e.j.s.Server Started @61370ms
   [junit4]   2> 53368 INFO  (jetty-launcher-26-thread-7) [    ] o.e.j.s.AbstractConnector Started ServerConnector@76fde472{HTTP/1.1,[http/1.1]}{127.0.0.1:57099}
   [junit4]   2> 53372 INFO  (jetty-launcher-26-thread-7) [    ] o.e.j.s.Server Started @61371ms
   [junit4]   2> 53369 INFO  (jetty-launcher-26-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@67e27c1b{HTTP/1.1,[http/1.1]}{127.0.0.1:60879}
   [junit4]   2> 53372 INFO  (jetty-launcher-26-thread-4) [    ] o.e.j.s.Server Started @61371ms
   [junit4]   2> 53369 INFO  (jetty-launcher-26-thread-5) [    ] o.e.j.s.AbstractConnector Started ServerConnector@69a22c35{HTTP/1.1,[http/1.1]}{127.0.0.1:32886}
   [junit4]   2> 53373 INFO  (jetty-launcher-26-thread-5) [    ] o.e.j.s.Server Started @61371ms
   [junit4]   2> 53369 INFO  (jetty-launcher-26-thread-6) [    ] o.e.j.s.AbstractConnector Started ServerConnector@766c2be9{HTTP/1.1,[http/1.1]}{127.0.0.1:42348}
   [junit4]   2> 53373 INFO  (jetty-launcher-26-thread-6) [    ] o.e.j.s.Server Started @61371ms
   [junit4]   2> 53369 INFO  (jetty-launcher-26-thread-10) [    ] o.e.j.s.AbstractConnector Started ServerConnector@18d1a429{HTTP/1.1,[http/1.1]}{127.0.0.1:33891}
   [junit4]   2> 53373 INFO  (jetty-launcher-26-thread-10) [    ] o.e.j.s.Server Started @61372ms
   [junit4]   2> 53369 INFO  (jetty-launcher-26-thread-9) [    ] o.e.j.s.AbstractConnector Started ServerConnector@274c3c6d{HTTP/1.1,[http/1.1]}{127.0.0.1:56645}
   [junit4]   2> 53376 INFO  (jetty-launcher-26-thread-9) [    ] o.e.j.s.Server Started @61375ms
   [junit4]   2> 53370 INFO  (jetty-launcher-26-thread-8) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7136fd51{HTTP/1.1,[http/1.1]}{127.0.0.1:55631}
   [junit4]   2> 53384 INFO  (jetty-launcher-26-thread-8) [    ] o.e.j.s.Server Started @61382ms
   [junit4]   2> 53372 INFO  (jetty-launcher-26-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58070}
   [junit4]   2> 53372 INFO  (jetty-launcher-26-thread-7) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57099}
   [junit4]   2> 53372 INFO  (jetty-launcher-26-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=60879}
   [junit4]   2> 53373 INFO  (jetty-launcher-26-thread-5) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=32886}
   [junit4]   2> 53373 INFO  (jetty-launcher-26-thread-6) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42348}
   [junit4]   2> 53373 INFO  (jetty-launcher-26-thread-10) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33891}
   [junit4]   2> 53376 INFO  (jetty-launcher-26-thread-9) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=56645}
   [junit4]   2> 53384 INFO  (jetty-launcher-26-thread-8) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55631}
   [junit4]   2> 53482 ERROR (jetty-launcher-26-thread-3) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 53482 INFO  (jetty-launcher-26-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 53482 INFO  (jetty-launcher-26-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 53483 INFO  (jetty-launcher-26-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 53491 ERROR (jetty-launcher-26-thread-8) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 53669 INFO  (jetty-launcher-26-thread-8) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 53670 INFO  (jetty-launcher-26-thread-8) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 53670 INFO  (jetty-launcher-26-thread-8) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 53491 ERROR (jetty-launcher-26-thread-9) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 53670 INFO  (jetty-launcher-26-thread-9) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 53670 INFO  (jetty-launcher-26-thread-9) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 53670 INFO  (jetty-launcher-26-thread-9) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 53491 ERROR (jetty-launcher-26-thread-10) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 53670 INFO  (jetty-launcher-26-thread-10) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 53670 INFO  (jetty-launcher-26-thread-10) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 53670 INFO  (jetty-launcher-26-thread-10) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 53491 ERROR (jetty-launcher-26-thread-6) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 53671 INFO  (jetty-launcher-26-thread-6) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 53671 INFO  (jetty-launcher-26-thread-6) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 53671 INFO  (jetty-launcher-26-thread-6) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 53492 ERROR (jetty-launcher-26-thread-5) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 53671 INFO  (jetty-launcher-26-thread-5) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 53671 INFO  (jetty-launcher-26-thread-5) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 53671 INFO  (jetty-launcher-26-thread-5) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 53492 ERROR (jetty-launcher-26-thread-4) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 53672 INFO  (jetty-launcher-26-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 53672 INFO  (jetty-launcher-26-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 53672 INFO  (jetty-launcher-26-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 53492 ERROR (jetty-launcher-26-thread-7) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 53672 INFO  (jetty-launcher-26-thread-7) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 53672 INFO  (jetty-launcher-26-thread-7) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 53672 INFO  (jetty-launcher-26-thread-7) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 53492 ERROR (jetty-launcher-26-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 53672 INFO  (jetty-launcher-26-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 53673 INFO  (jetty-launcher-26-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 53673 INFO  (jetty-launcher-26-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 53691 ERROR (jetty-launcher-26-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 53691 INFO  (jetty-launcher-26-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 53691 INFO  (jetty-launcher-26-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 53691 INFO  (jetty-launcher-26-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 53721 INFO  (jetty-launcher-26-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-18T01:50:03.776Z
   [junit4]   2> 53723 INFO  (jetty-launcher-26-thread-6) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-18T01:50:03.754Z
   [junit4]   2> 53723 INFO  (jetty-launcher-26-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-18T01:50:03.568Z
   [junit4]   2> 53728 INFO  (jetty-launcher-26-thread-5) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-18T01:50:03.754Z
   [junit4]   2> 53732 INFO  (jetty-launcher-26-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-18T01:50:03.755Z
   [junit4]   2> 53732 INFO  (jetty-launcher-26-thread-9) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-18T01:50:03.753Z
   [junit4]   2> 53732 INFO  (jetty-launcher-26-thread-10) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-18T01:50:03.753Z
   [junit4]   2> 53733 INFO  (jetty-launcher-26-thread-7) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-18T01:50:03.755Z
   [junit4]   2> 53734 INFO  (jetty-launcher-26-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-18T01:50:03.756Z
   [junit4]   2> 53734 INFO  (jetty-launcher-26-thread-8) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-04-18T01:50:03.753Z
   [junit4]   2> 53736 INFO  (jetty-launcher-26-thread-2) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 54284 INFO  (jetty-launcher-26-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 54284 INFO  (jetty-launcher-26-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 54284 INFO  (jetty-launcher-26-thread-7) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 54287 INFO  (jetty-launcher-26-thread-5) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 54292 INFO  (jetty-launcher-26-thread-6) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 54362 INFO  (jetty-launcher-26-thread-10) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 54363 INFO  (jetty-launcher-26-thread-9) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 54397 INFO  (jetty-launcher-26-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 54409 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b7ec092cb0009, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 54410 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b7ec092cb0003, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 54410 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b7ec092cb0004, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 54410 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b7ec092cb0007, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 54411 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b7ec092cb000a, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 54411 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b7ec092cb0005, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 54411 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b7ec092cb0001, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 54458 INFO  (jetty-launcher-26-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 54458 INFO  (jetty-launcher-26-thread-8) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 54636 INFO  (jetty-launcher-26-thread-10) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46794/solr
   [junit4]   2> 54660 INFO  (jetty-launcher-26-thread-6) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46794/solr
   [junit4]   2> 54907 INFO  (jetty-launcher-26-thread-8) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46794/solr
   [junit4]   2> 54961 INFO  (jetty-launcher-26-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46794/solr
   [junit4]   2> 55070 INFO  (jetty-launcher-26-thread-5) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46794/solr
   [junit4]   2> 55145 INFO  (jetty-launcher-26-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46794/solr
   [junit4]   2> 55191 INFO  (jetty-launcher-26-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46794/solr
   [junit4]   2> 55232 INFO  (jetty-launcher-26-thread-9) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46794/solr
   [junit4]   2> 55250 INFO  (jetty-launcher-26-thread-7) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46794/solr
   [junit4]   2> 55197 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b7ec092cb000d, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 54961 INFO  (jetty-launcher-26-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46794/solr
   [junit4]   2> 55354 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b7ec092cb000c, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 55355 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b7ec092cb000f, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 55734 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b7ec092cb0010, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 55782 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15b7ec092cb0016, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 55790 INFO  (jetty-launcher-26-thread-8) [n:127.0.0.1:55631_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 55791 INFO  (jetty-launcher-26-thread-2) [n:127.0.0.1:59189_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 55792 INFO  (jetty-launcher-26-thread-10) [n:127.0.0.1:33891_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 55794 INFO  (jetty-launcher-26-thread-8) [n:127.0.0.1:55631_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55631_solr
   [junit4]   2> 55799 INFO  (jetty-launcher-26-thread-2) [n:127.0.0.1:59189_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59189_solr
   [junit4]   2> 55800 INFO  (jetty-launcher-26-thread-10) [n:127.0.0.1:33891_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33891_solr
   [junit4]   2> 55804 INFO  (jetty-launcher-26-thread-1) [n:127.0.0.1:58070_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 55808 INFO  (jetty-launcher-26-thread-1) [n:127.0.0.1:58070_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 55824 INFO  (zkCallback-68-thread-1-processing-n:127.0.0.1:59189_solr) [n:127.0.0.1:59189_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 55884 INFO  (zkCallback-65-thread-1-processing-n:127.0.0.1:33891_solr) [n:127.0.0.1:33891_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 55902 INFO  (jetty-launcher-26-thread-6) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 55905 INFO  (jetty-launcher-26-thread-6) [n:127.0.0.1:42348_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 55923 INFO  (jetty-launcher-26-thread-1) [n:127.0.0.1:58070_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58070_solr
   [junit4]   2> 55924 INFO  (jetty-launcher-26-thread-8) [n:127.0.0.1:55631_solr    ] o.a.s.c.Overseer Overseer (id=97811182480195604-127.0.0.1:55631_solr-n_0000000000) starting
   [junit4]   2> 55935 INFO  (zkCallback-68-thread-1-processing-n:127.0.0.1:59189_solr) [n:127.0.0.1:59189_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 55941 INFO  (zkCallback-65-thread-1-processing-n:127.0.0.1:33891_solr) [n:127.0.0.1:33891_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 55946 INFO  (jetty-launcher-26-thread-6) [n:127.0.0.1:42348_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42348_solr
   [junit4]   2> 55960 INFO  (zkCallback-68-thread-1-processing-n:127.0.0.1:59189_solr) [n:127.0.0.1:59189_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 55961 INFO  (zkCallback-65-thread-1-processing-n:127.0.0.1:33891_solr) [n:127.0.0.1:33891_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 55970 INFO  (jetty-launcher-26-thread-5) [n:127.0.0.1:32886_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 55973 INFO  (jetty-launcher-26-thread-3) [n:127.0.0.1:59203_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 55994 INFO  (zkCallback-75-thread-1-processing-n:127.0.0.1:58070_solr) [n:127.0.0.1:58070_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (4)
   [junit4]   2> 55994 INFO  (zkCallback-67-thread-1-processing-n:127.0.0.1:55631_solr) [n:127.0.0.1:55631_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 55994 INFO  (zkCallback-72-thread-1-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 56010 INFO  (zkCallback-67-thread-2-processing-n:127.0.0.1:55631_solr) [n:127.0.0.1:55631_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
   [junit4]   2> 56012 INFO  (jetty-launcher-26-thread-5) [n:127.0.0.1:32886_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 56013 INFO  (jetty-launcher-26-thread-9) [n:127.0.0.1:56645_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 56017 INFO  (jetty-launcher-26-thread-3) [n:127.0.0.1:59203_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 56130 INFO  (jetty-launcher-26-thread-5) [n:127.0.0.1:32886_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32886_solr
   [junit4]   2> 56133 INFO  (jetty-launcher-26-thread-3) [n:127.0.0.1:59203_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59203_solr
   [junit4]   2> 56135 INFO  (jetty-launcher-26-thread-9) [n:127.0.0.1:56645_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 56135 INFO  (zkCallback-68-thread-1-processing-n:127.0.0.1:59189_solr) [n:127.0.0.1:59189_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 56135 INFO  (zkCallback-65-thread-1-processing-n:127.0.0.1:33891_solr) [n:127.0.0.1:33891_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 56144 INFO  (zkCallback-72-thread-2-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (5)
   [junit4]   2> 56147 INFO  (zkCallback-68-thread-1-processing-n:127.0.0.1:59189_solr) [n:127.0.0.1:59189_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
   [junit4]   2> 56147 INFO  (zkCallback-65-thread-1-processing-n:127.0.0.1:33891_solr) [n:127.0.0.1:33891_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
   [junit4]   2> 56152 INFO  (jetty-launcher-26-thread-7) [n:127.0.0.1:57099_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (6)
   [junit4]   2> 56156 INFO  (jetty-launcher-26-thread-9) [n:127.0.0.1:56645_solr    ] o.a.s.c.ZkController Register node as live in ZooK