[JENKINS] Lucene-Solr-NightlyTests-7.0 - Build # 1 - Failure

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-NightlyTests-7.0 - Build # 1 - Failure

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.0/1/

No tests ran.

Build Log:
[...truncated 78 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/build.xml:818: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/lucene/common-build.xml:422: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/lucene/common-build.xml:459: Ivy is not available

Total time: 0 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-NightlyTests-7.0 - Build # 2 - Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.0/2/

4 tests failed.
FAILED:  org.apache.solr.schema.TestPointFields.testDoublePointStats

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
        at __randomizedtesting.SeedInfo.seed([FDC2A906B090758:20C6E4C69FA41BFC]:0)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:878)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:845)
        at org.apache.solr.schema.TestPointFields.testPointStats(TestPointFields.java:2200)
        at org.apache.solr.schema.TestPointFields.testDoublePointStats(TestPointFields.java:689)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//*[@numFound='11']
        xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="21" start="0"><doc><str name="id">0</str><double name="number_p_d_dv">-0.2651145</double></doc><doc><str name="id">1</str><double name="number_p_d_dv">-82882.07</double></doc><doc><str name="id">2</str><double name="number_p_d_dv">-126.417</double></doc><doc><str name="id">3</str><double name="number_p_d_dv">7193997.0</double></doc><doc><str name="id">4</str><double name="number_p_d_dv">-0.4656807</double></doc><doc><str name="id">5</str><double name="number_p_d_dv">-8842.107</double></doc><doc><str name="id">6</str><double name="number_p_d_dv">46.33978</double></doc><doc><str name="id">7</str><double name="number_p_d_dv">0.8667498</double></doc><doc><str name="id">8</str><double name="number_p_d_dv">218757.9</double></doc><doc><str name="id">9</str><double name="number_p_d_dv">5172158.0</double></doc></result><lst name="stats"><lst name="stats_fields"><lst name="number_p_d_dv"><double name="min">-82882.07</double><double name="max">7193997.0</double><long name="count">20</long><long name="missing">1</long><double name="sum">1.2490645725103797E7</double><double name="sumOfSquares">7.855961768125258E13</double><double name="mean">624532.2862551899</double><double name="stddev">1929805.0917364943</double></lst></lst></lst>
</response>

        request was:q=*:*&stats=true&fl=id,+number_p_d_dv&wt=xml&stats.field=number_p_d_dv
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:871)
        ... 42 more


FAILED:  org.apache.solr.schema.TestPointFields.testFloatPointStats

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
        at __randomizedtesting.SeedInfo.seed([FDC2A906B090758:B5D652B7D318C5BA]:0)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:878)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:845)
        at org.apache.solr.schema.TestPointFields.testPointStats(TestPointFields.java:2200)
        at org.apache.solr.schema.TestPointFields.testFloatPointStats(TestPointFields.java:989)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//*[@numFound='11']
        xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="21" start="0"><doc><str name="id">0</str><float name="number_p_f_dv">-5.77594</float></doc><doc><str name="id">1</str><float name="number_p_f_dv">0.2209248</float></doc><doc><str name="id">2</str><float name="number_p_f_dv">306.6558</float></doc><doc><str name="id">3</str><float name="number_p_f_dv">-2.373698</float></doc><doc><str name="id">4</str><float name="number_p_f_dv">355.7655</float></doc><doc><str name="id">5</str><float name="number_p_f_dv">-864999.4</float></doc><doc><str name="id">6</str><float name="number_p_f_dv">-47338.32</float></doc><doc><str name="id">7</str><float name="number_p_f_dv">0.7183317</float></doc><doc><str name="id">8</str><float name="number_p_f_dv">-5175.803</float></doc><doc><str name="id">9</str><float name="number_p_f_dv">0.1814456</float></doc></result><lst name="stats"><lst name="stats_fields"><lst name="number_p_f_dv"><double name="min">-864999.375</double><double name="max">6743908.0</double><long name="count">20</long><long name="missing">1</long><double name="sum">8885599.336620271</double><double name="sumOfSquares">5.5567164356464664E13</double><double name="mean">444279.96683101356</double><double name="stddev">1648276.142199182</double></lst></lst></lst>
</response>

        request was:q=*:*&stats=true&fl=id,+number_p_f_dv&wt=xml&stats.field=number_p_f_dv
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:871)
        ... 42 more


FAILED:  org.apache.solr.schema.TestPointFields.testLongPointStats

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
        at __randomizedtesting.SeedInfo.seed([FDC2A906B090758:B821BC45405CF33D]:0)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:878)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:845)
        at org.apache.solr.schema.TestPointFields.testPointStats(TestPointFields.java:2200)
        at org.apache.solr.schema.TestPointFields.testLongPointStats(TestPointFields.java:1251)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//*[@numFound='11']
        xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="21" start="0"><doc><str name="id">0</str><long name="number_p_l_dv">-1220518</long></doc><doc><str name="id">1</str><long name="number_p_l_dv">3333406</long></doc><doc><str name="id">2</str><long name="number_p_l_dv">1152950</long></doc><doc><str name="id">3</str><long name="number_p_l_dv">3095391</long></doc><doc><str name="id">4</str><long name="number_p_l_dv">6488241</long></doc><doc><str name="id">5</str><long name="number_p_l_dv">5997323</long></doc><doc><str name="id">6</str><long name="number_p_l_dv">-1466616</long></doc><doc><str name="id">7</str><long name="number_p_l_dv">5290549</long></doc><doc><str name="id">8</str><long name="number_p_l_dv">-3664392</long></doc><doc><str name="id">9</str><long name="number_p_l_dv">9615667</long></doc></result><lst name="stats"><lst name="stats_fields"><lst name="number_p_l_dv"><double name="min">-8285236.0</double><double name="max">9615667.0</double><long name="count">20</long><long name="missing">1</long><double name="sum">1154663.0</double><double name="sumOfSquares">5.70470358525951E14</double><double name="mean">57733.15</double><double name="stddev">5479164.823957507</double></lst></lst></lst>
</response>

        request was:q=*:*&stats=true&fl=id,+number_p_l_dv&wt=xml&stats.field=number_p_l_dv
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:871)
        ... 42 more


FAILED:  org.apache.solr.schema.TestPointFields.testIntPointStats

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
        at __randomizedtesting.SeedInfo.seed([FDC2A906B090758:6459D39BF5AF1981]:0)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:878)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:845)
        at org.apache.solr.schema.TestPointFields.testPointStats(TestPointFields.java:2200)
        at org.apache.solr.schema.TestPointFields.testIntPointStats(TestPointFields.java:278)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//*[@numFound='11']
        xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="21" start="0"><doc><str name="id">0</str><int name="number_p_i_dv">-9709620</int></doc><doc><str name="id">1</str><int name="number_p_i_dv">6943689</int></doc><doc><str name="id">2</str><int name="number_p_i_dv">-9141310</int></doc><doc><str name="id">3</str><int name="number_p_i_dv">8593189</int></doc><doc><str name="id">4</str><int name="number_p_i_dv">-5053854</int></doc><doc><str name="id">5</str><int name="number_p_i_dv">-4933745</int></doc><doc><str name="id">6</str><int name="number_p_i_dv">-6879691</int></doc><doc><str name="id">7</str><int name="number_p_i_dv">-6439933</int></doc><doc><str name="id">8</str><int name="number_p_i_dv">914792</int></doc><doc><str name="id">9</str><int name="number_p_i_dv">4509951</int></doc></result><lst name="stats"><lst name="stats_fields"><lst name="number_p_i_dv"><double name="min">-9709620.0</double><double name="max">8593189.0</double><long name="count">20</long><long name="missing">1</long><double name="sum">-4.2208598E7</double><double name="sumOfSquares">7.29546509657556E14</double><double name="mean">-2110429.9</double><double name="stddev">5805932.639171201</double></lst></lst></lst>
</response>

        request was:q=*:*&stats=true&fl=id,+number_p_i_dv&wt=xml&stats.field=number_p_i_dv
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:871)
        ... 42 more




Build Log:
[...truncated 13496 lines...]
   [junit4] Suite: org.apache.solr.schema.TestPointFields
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J1/temp/solr.schema.TestPointFields_FDC2A906B090758-001/init-core-data-001
   [junit4]   2> 4048823 WARN  (SUITE-TestPointFields-seed#[FDC2A906B090758]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=14 numCloses=14
   [junit4]   2> 4048823 INFO  (SUITE-TestPointFields-seed#[FDC2A906B090758]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 4048824 INFO  (SUITE-TestPointFields-seed#[FDC2A906B090758]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 4048825 INFO  (SUITE-TestPointFields-seed#[FDC2A906B090758]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 4048825 INFO  (SUITE-TestPointFields-seed#[FDC2A906B090758]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 4048857 INFO  (SUITE-TestPointFields-seed#[FDC2A906B090758]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 4048883 INFO  (SUITE-TestPointFields-seed#[FDC2A906B090758]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=schema-point
   [junit4]   2> 4048891 INFO  (SUITE-TestPointFields-seed#[FDC2A906B090758]-worker) [    ] o.a.s.s.IndexSchema Loaded schema schema-point/1.6 with uniqueid field id
   [junit4]   2> 4048925 INFO  (SUITE-TestPointFields-seed#[FDC2A906B090758]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3376a754
   [junit4]   2> 4048930 INFO  (SUITE-TestPointFields-seed#[FDC2A906B090758]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3376a754
   [junit4]   2> 4048930 INFO  (SUITE-TestPointFields-seed#[FDC2A906B090758]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3376a754
   [junit4]   2> 4048932 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 4049020 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 4049066 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=schema-point
   [junit4]   2> 4049075 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema schema-point/1.6 with uniqueid field id
   [junit4]   2> 4049075 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 4049075 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3376a754
   [junit4]   2> 4049075 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 4049076 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J1/temp/solr.schema.TestPointFields_FDC2A906B090758-001/init-core-data-001/]
   [junit4]   2> 4049088 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=20, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.65234375, noCFSRatio=0.0]
   [junit4]   2> 4049202 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 4049202 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 4049203 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 4049203 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 4049204 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=986240529, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 4049204 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7f5f7fe8[collection1] main]
   [junit4]   2> 4049205 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 4049257 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 4049354 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 4049355 INFO  (searcherExecutor-15586-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 4049355 INFO  (searcherExecutor-15586-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 4049355 INFO  (searcherExecutor-15586-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 4049355 INFO  (searcherExecutor-15586-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 4049356 INFO  (searcherExecutor-15586-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 4049356 INFO  (searcherExecutor-15586-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 4049356 INFO  (searcherExecutor-15586-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 4049356 INFO  (searcherExecutor-15586-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 4049357 INFO  (searcherExecutor-15586-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 4049358 INFO  (searcherExecutor-15586-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7f5f7fe8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4049358 INFO  (coreLoadExecutor-15585-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1572663928488460288
   [junit4]   2> 4049362 INFO  (SUITE-TestPointFields-seed#[FDC2A906B090758]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 4049366 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDoublePointMultiValuedFunctionQuery
   [junit4]   2> 4049368 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0 (1572663928497897472)]} 0 1
   [junit4]   2> 4049370 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1 (1572663928501043200)]} 0 0
   [junit4]   2> 4049371 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2 (1572663928502091776)]} 0 0
   [junit4]   2> 4049371 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3 (1572663928502091777)]} 0 0
   [junit4]   2> 4049371 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[4 (1572663928502091778)]} 0 0
   [junit4]   2> 4049372 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[5 (1572663928503140352)]} 0 0
   [junit4]   2> 4049372 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[6 (1572663928503140353)]} 0 0
   [junit4]   2> 4049373 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[7 (1572663928504188928)]} 0 0
   [junit4]   2> 4049373 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[8 (1572663928504188929)]} 0 0
   [junit4]   2> 4049373 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[9 (1572663928504188930)]} 0 0
   [junit4]   2> 4049374 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1572663928505237504,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 4049374 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6fb478c3 commitCommandVersion:1572663928505237504
   [junit4]   2> 4049430 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@70746b60[collection1] main]
   [junit4]   2> 4049433 INFO  (searcherExecutor-15586-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@70746b60[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):c10)))}
   [junit4]   2> 4049433 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 4049433 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 59
   [junit4]   2> 4049435 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&fl=id,+number_p_d_mv_dv&sort=field(number_p_d_mv_dv,+min)+desc&wt=xml} hits=10 status=0 QTime=1
   [junit4]   2> 4049449 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.h.RequestHandlerBase Ignoring exception matching .
   [junit4]   2> 4049449 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&fl=id&sort=field(number_p_d_mv,min)+desc&wt=xml} status=400 QTime=13
   [junit4]   2> 4049450 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.h.RequestHandlerBase Ignoring exception matching .
   [junit4]   2> 4049450 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&fl=id,+field(number_p_d_mv,min)&wt=xml} status=400 QTime=0
   [junit4]   2> 4049450 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.h.RequestHandlerBase Ignoring exception matching .
   [junit4]   2> 4049450 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&fl=id,+field(number_p_d_mv_dv,foo)&wt=xml} status=400 QTime=0
   [junit4]   2> 4049456 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0 (1572663928589123584)]} 0 1
   [junit4]   2> 4049456 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1 (1572663928591220736)]} 0 0
   [junit4]   2> 4049457 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2 (1572663928592269312)]} 0 0
   [junit4]   2> 4049457 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3 (1572663928592269313)]} 0 0
   [junit4]   2> 4049458 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[4 (1572663928592269314)]} 0 0
   [junit4]   2> 4049458 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[5 (1572663928593317888)]} 0 0
   [junit4]   2> 4049459 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[6 (1572663928593317889)]} 0 0
   [junit4]   2> 4049459 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[7 (1572663928594366464)]} 0 0
   [junit4]   2> 4049460 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[8 (1572663928594366465)]} 0 0
   [junit4]   2> 4049460 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[9 (1572663928595415040)]} 0 0
   [junit4]   2> 4049460 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1572663928595415041,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 4049460 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6fb478c3 commitCommandVersion:1572663928595415041
   [junit4]   2> 4049477 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@538961a9[collection1] main]
   [junit4]   2> 4049486 INFO  (searcherExecutor-15586-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@538961a9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1(7.0.0):c10)))}
   [junit4]   2> 4049486 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 4049487 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 26
   [junit4]   2> 4049488 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&fl=id,+number_p_d_mv_dv&sort=field(number_p_d_mv_dv,+min)+desc&wt=xml} hits=10 status=0 QTime=0
   [junit4]   2> 4049489 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.h.RequestHandlerBase Ignoring exception matching .
   [junit4]   2> 4049489 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&fl=id&sort=field(number_p_d_mv,min)+desc&wt=xml} status=400 QTime=0
   [junit4]   2> 4049507 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.h.RequestHandlerBase Ignoring exception matching .
   [junit4]   2> 4049507 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&fl=id,+field(number_p_d_mv,min)&wt=xml} status=400 QTime=17
   [junit4]   2> 4049507 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.h.RequestHandlerBase Ignoring exception matching .
   [junit4]   2> 4049507 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&fl=id,+field(number_p_d_mv_dv,foo)&wt=xml} status=400 QTime=0
   [junit4]   2> 4049508 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 4049508 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@dab5dd1[collection1] realtime]
   [junit4]   2> 4049508 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 0
   [junit4]   2> 4049509 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1572663928646795264,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 4049509 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6fb478c3 commitCommandVersion:1572663928646795264
   [junit4]   2> 4049509 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@6c8964eb[collection1] main]
   [junit4]   2> 4049512 INFO  (searcherExecutor-15586-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6c8964eb[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4049512 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 4049512 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 3
   [junit4]   2> 4049512 INFO  (TEST-TestPointFields.testDoublePointMultiValuedFunctionQuery-seed#[FDC2A906B090758]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDoublePointMultiValuedFunctionQuery
   [junit4]   2> 4049516 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDoublePointFieldExactQuery
   [junit4]   2> 4049521 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0 (1572663928656232448)]} 0 2
   [junit4]   2> 4049521 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1 (1572663928659378176)]} 0 0
   [junit4]   2> 4049522 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2 (1572663928660426752)]} 0 0
   [junit4]   2> 4049522 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3 (1572663928660426753)]} 0 0
   [junit4]   2> 4049523 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[4 (1572663928660426754)]} 0 0
   [junit4]   2> 4049523 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[5 (1572663928661475328)]} 0 0
   [junit4]   2> 4049523 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[6 (1572663928661475329)]} 0 0
   [junit4]   2> 4049524 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[7 (1572663928662523904)]} 0 0
   [junit4]   2> 4049524 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[8 (1572663928662523905)]} 0 0
   [junit4]   2> 4049525 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[9 (1572663928662523906)]} 0 0
   [junit4]   2> 4049525 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[10 (1572663928663572480)]} 0 0
   [junit4]   2> 4049525 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[11 (1572663928663572481)]} 0 0
   [junit4]   2> 4049526 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[12 (1572663928664621056)]} 0 0
   [junit4]   2> 4049526 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[13 (1572663928664621057)]} 0 0
   [junit4]   2> 4049526 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[14 (1572663928664621058)]} 0 0
   [junit4]   2> 4049527 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[15 (1572663928665669632)]} 0 0
   [junit4]   2> 4049527 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[16 (1572663928665669633)]} 0 0
   [junit4]   2> 4049528 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[17 (1572663928665669634)]} 0 0
   [junit4]   2> 4049528 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[18 (1572663928666718208)]} 0 0
   [junit4]   2> 4049528 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[19 (1572663928666718209)]} 0 0
   [junit4]   2> 4049529 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1572663928667766784,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 4049529 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6fb478c3 commitCommandVersion:1572663928667766784
   [junit4]   2> 4049531 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@6ec835dd[collection1] main]
   [junit4]   2> 4049531 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 4049533 INFO  (searcherExecutor-15586-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6ec835dd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2(7.0.0):c20)))}
   [junit4]   2> 4049533 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 4049535 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:6.857243780360694E-188&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049535 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:\-3.7551649354501467E-7&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049536 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:8.341607406708468E250&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049536 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:1.914572233997143E-51&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049537 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:\-6.27037997475455E86&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049538 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:2.5756669546081176E280&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049538 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:2.7707650501254253E21&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049539 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:2.6861298928818053E-289&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049539 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:1.0191817390165676E290&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049540 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:\-6.01842461034377E-166&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049540 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:9.456200410166227E161&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049541 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:\-1.5924979555364308E-29&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049541 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:6.726945277806774E273&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049541 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:\-3.1323637893731433E-47&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049542 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:1.5355279979503967E-271&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049542 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:1.708479124521285E-21&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049543 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:\-7.46412476188314E-133&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049543 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:\-1.0862328593238663E274&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049544 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:\-7.989048720348522E279&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049544 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:\-1.6313417304257326E-264&fl=id,number_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049559 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_d:(\-1.0862328593238663E274+OR+2.6861298928818053E-289+OR+1.914572233997143E-51+OR+8.341607406708468E250+OR+\-7.46412476188314E-133+OR+9.456200410166227E161+OR+\-1.5924979555364308E-29+OR+\-7.989048720348522E279+OR+1.0191817390165676E290+OR+1.5355279979503967E-271+OR+6.726945277806774E273+OR+\-6.01842461034377E-166+OR+\-1.6313417304257326E-264+OR+\-3.1323637893731433E-47+OR+2.7707650501254253E21+OR+\-3.7551649354501467E-7+OR+2.5756669546081176E280+OR+1.708479124521285E-21+OR+6.857243780360694E-188+OR+\-6.27037997475455E86)&debug=true&wt=xml} hits=20 status=0 QTime=13
   [junit4]   2> 4049561 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 4049561 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@68a2918b[collection1] realtime]
   [junit4]   2> 4049561 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 0
   [junit4]   2> 4049561 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1572663928701321216,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 4049562 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6fb478c3 commitCommandVersion:1572663928701321216
   [junit4]   2> 4049562 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@2802919d[collection1] main]
   [junit4]   2> 4049564 INFO  (searcherExecutor-15586-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2802919d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4049564 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 4049564 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 3
   [junit4]   2> 4049566 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0 (1572663928705515520)]} 0 1
   [junit4]   2> 4049566 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1 (1572663928706564096)]} 0 0
   [junit4]   2> 4049567 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2 (1572663928707612672)]} 0 0
   [junit4]   2> 4049567 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3 (1572663928707612673)]} 0 0
   [junit4]   2> 4049568 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[4 (1572663928708661248)]} 0 0
   [junit4]   2> 4049568 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[5 (1572663928708661249)]} 0 0
   [junit4]   2> 4049569 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[6 (1572663928708661250)]} 0 0
   [junit4]   2> 4049569 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[7 (1572663928709709824)]} 0 0
   [junit4]   2> 4049569 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[8 (1572663928709709825)]} 0 0
   [junit4]   2> 4049570 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[9 (1572663928710758400)]} 0 0
   [junit4]   2> 4049570 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[10 (1572663928710758401)]} 0 0
   [junit4]   2> 4049570 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[11 (1572663928710758402)]} 0 0
   [junit4]   2> 4049571 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[12 (1572663928711806976)]} 0 0
   [junit4]   2> 4049571 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[13 (1572663928711806977)]} 0 0
   [junit4]   2> 4049572 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[14 (1572663928711806978)]} 0 0
   [junit4]   2> 4049572 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[15 (1572663928712855552)]} 0 0
   [junit4]   2> 4049572 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[16 (1572663928712855553)]} 0 0
   [junit4]   2> 4049573 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[17 (1572663928713904128)]} 0 0
   [junit4]   2> 4049578 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[18 (1572663928719147008)]} 0 4
   [junit4]   2> 4049578 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[19 (1572663928719147009)]} 0 0
   [junit4]   2> 4049579 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1572663928720195584,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 4049579 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6fb478c3 commitCommandVersion:1572663928720195584
   [junit4]   2> 4049588 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@196de787[collection1] main]
   [junit4]   2> 4049591 INFO  (searcherExecutor-15586-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@196de787[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_3(7.0.0):c20)))}
   [junit4]   2> 4049591 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 4049591 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 12
   [junit4]   2> 4049593 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:3.733292141645897E220&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=1
   [junit4]   2> 4049593 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:2.996603044010409E-13&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049594 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:1.7692174464708838E274&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049594 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:\-2.3151324057542722E257&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049595 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:\-1.1214781642089957E189&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049596 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:3.1568555258504556E112&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049596 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:\-5.501490820905642E-20&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049597 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:1.832553494430207E277&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049598 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:2.454863961506394E165&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049598 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:2.1699118801720638E-92&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049599 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:\-1.9580145987630187E-51&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049600 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:9.358995322843673E-146&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049616 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:3.529379187089641E-34&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=16
   [junit4]   2> 4049617 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:\-1.1376017980857958E128&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049618 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:1.403610673498054E220&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049618 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:\-7.217143765975271E-269&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049619 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:\-8.088720712646947E108&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049620 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:\-1.795813059068012E229&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049620 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:\-6.767276928751769E-74&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049621 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:3.2978960496668717E-268&fl=id,number_p_d&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049640 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d:(3.2978960496668717E-268+OR+2.1699118801720638E-92+OR+9.358995322843673E-146+OR+\-2.3151324057542722E257+OR+\-7.217143765975271E-269+OR+2.996603044010409E-13+OR+\-1.1376017980857958E128+OR+3.529379187089641E-34+OR+2.454863961506394E165+OR+\-1.1214781642089957E189+OR+\-1.795813059068012E229+OR+1.7692174464708838E274+OR+1.832553494430207E277+OR+3.1568555258504556E112+OR+\-1.9580145987630187E-51+OR+\-6.767276928751769E-74+OR+\-5.501490820905642E-20+OR+\-8.088720712646947E108+OR+1.403610673498054E220+OR+3.733292141645897E220)&debug=true&wt=xml} hits=20 status=0 QTime=18
   [junit4]   2> 4049641 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 4049642 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@2cf9fcc0[collection1] realtime]
   [junit4]   2> 4049642 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 1
   [junit4]   2> 4049642 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1572663928786255872,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 4049643 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6fb478c3 commitCommandVersion:1572663928786255872
   [junit4]   2> 4049643 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@38b6ed96[collection1] main]
   [junit4]   2> 4049646 INFO  (searcherExecutor-15586-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@38b6ed96[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4049658 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 4049658 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 15
   [junit4]   2> 4049670 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0 (1572663928808275968)]} 0 6
   [junit4]   2> 4049670 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1 (1572663928815616000)]} 0 0
   [junit4]   2> 4049671 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2 (1572663928816664576)]} 0 0
   [junit4]   2> 4049671 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3 (1572663928816664577)]} 0 0
   [junit4]   2> 4049671 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[4 (1572663928816664578)]} 0 0
   [junit4]   2> 4049672 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[5 (1572663928817713152)]} 0 0
   [junit4]   2> 4049672 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[6 (1572663928817713153)]} 0 0
   [junit4]   2> 4049673 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[7 (1572663928818761728)]} 0 0
   [junit4]   2> 4049673 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[8 (1572663928818761729)]} 0 0
   [junit4]   2> 4049673 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[9 (1572663928818761730)]} 0 0
   [junit4]   2> 4049674 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[10 (1572663928819810304)]} 0 0
   [junit4]   2> 4049674 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[11 (1572663928819810305)]} 0 0
   [junit4]   2> 4049675 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[12 (1572663928819810306)]} 0 0
   [junit4]   2> 4049675 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[13 (1572663928820858880)]} 0 0
   [junit4]   2> 4049675 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[14 (1572663928820858881)]} 0 0
   [junit4]   2> 4049676 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[15 (1572663928821907456)]} 0 0
   [junit4]   2> 4049676 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[16 (1572663928821907457)]} 0 0
   [junit4]   2> 4049682 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[17 (1572663928827150336)]} 0 0
   [junit4]   2> 4049688 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[18 (1572663928828198912)]} 0 5
   [junit4]   2> 4049689 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[19 (1572663928834490368)]} 0 0
   [junit4]   2> 4049689 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1572663928835538944,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 4049689 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6fb478c3 commitCommandVersion:1572663928835538944
   [junit4]   2> 4049693 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@43ffdbc3[collection1] main]
   [junit4]   2> 4049695 INFO  (searcherExecutor-15586-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@43ffdbc3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_4(7.0.0):c20)))}
   [junit4]   2> 4049696 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 4049696 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 6
   [junit4]   2> 4049697 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:\-3.066543263534658E-176&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049697 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:\-1.1177830699420078E-112&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049698 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:8.910608453735807E173&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049698 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:\-4.054463729553171E210&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049699 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:\-2.7391381825071164E286&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049700 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:\-2.820921347974469E237&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049706 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:\-5.879145199042671E218&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049707 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:1.0315143727030086E169&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049707 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:6.13755445776007E-66&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049708 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:389.3403683805502&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049708 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:\-1.924701226269316E-51&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049729 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:2.0610127555424197E77&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049730 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:1.4990954504237726E-53&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049730 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:3.842909685732954E143&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049731 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:\-2.525737921574324E151&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049731 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:3.6015296939364154E-132&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049732 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:7.993438725246967E101&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049732 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:\-3.6015650843510193E171&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049733 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:\-3.8618245911719494E90&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049734 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:\-3.62549647631328E-206&fl=id,number_p_d_mv&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 4049747 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=number_p_d_mv:(\-3.8618245911719494E90+OR+\-5.879145199042671E218+OR+3.6015296939364154E-132+OR+\-2.525737921574324E151+OR+1.4990954504237726E-53+OR+\-3.6015650843510193E171+OR+389.3403683805502+OR+1.0315143727030086E169+OR+8.910608453735807E173+OR+6.13755445776007E-66+OR+3.842909685732954E143+OR+\-1.924701226269316E-51+OR+2.0610127555424197E77+OR+\-2.7391381825071164E286+OR+\-3.62549647631328E-206+OR+\-2.820921347974469E237+OR+7.993438725246967E101+OR+\-4.054463729553171E210+OR+\-1.1177830699420078E-112+OR+\-3.066543263534658E-176)&debug=true&wt=xml} hits=20 status=0 QTime=12
   [junit4]   2> 4049748 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 4049748 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@4b1a372b[collection1] realtime]
   [junit4]   2> 4049749 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 0
   [junit4]   2> 4049749 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1572663928898453504,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 4049749 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6fb478c3 commitCommandVersion:1572663928898453504
   [junit4]   2> 4049801 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@8815da5[collection1] main]
   [junit4]   2> 4049801 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 4049803 INFO  (searcherExecutor-15586-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@8815da5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4049803 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 54
   [junit4]   2> 4049805 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0 (1572663928956125184)]} 0 1
   [junit4]   2> 4049805 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1 (1572663928957173760)]} 0 0
   [junit4]   2> 4049806 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2 (1572663928958222336)]} 0 0
   [junit4]   2> 4049806 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3 (1572663928958222337)]} 0 0
   [junit4]   2> 4049807 INFO  (TEST-TestPointFields.testDoublePointFieldExactQuery-seed#[FDC2A906B090758]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]

[...truncated too long message...]

slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J1/temp/solr.schema.TestPointFields_FDC2A906B090758-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {number_d=PostingsFormat(name=LuceneVarGapDocFreqInterval), _version_=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=PostingsFormat(name=LuceneFixedGap)}, docValues:{foo_p_f_ni_dv_ns=DocValuesFormat(name=Asserting), foo_p_f_ni_dv_ns_mv=DocValuesFormat(name=Lucene70), number_p_dt_dv_ns=DocValuesFormat(name=Memory), foo_p_d_ni_dv_ns_mv=DocValuesFormat(name=Asserting), foo_p_i_ni_dv_ns=DocValuesFormat(name=Memory), number_p_f_ni_mv_dv_smf=DocValuesFormat(name=Asserting), number_p_dt_ni_mv_dv_sml=DocValuesFormat(name=Lucene70), number_p_l_ni_dv=DocValuesFormat(name=Memory), number_p_f_dv=DocValuesFormat(name=Memory), number_p_i_dv_ns=DocValuesFormat(name=Asserting), number_p_l_dv_ns=DocValuesFormat(name=Memory), foo_p_l_ni_dv_ns=DocValuesFormat(name=Lucene70), number_p_dt_ni_mv_dv=DocValuesFormat(name=Memory), number_p_l_dv_sml=DocValuesFormat(name=Lucene70), number_p_i_dv_smf=DocValuesFormat(name=Lucene70), number_p_d_ni_ns_dv=DocValuesFormat(name=Lucene70), number_p_dt_ni_dv_ns_mv=DocValuesFormat(name=Memory), number_p_i_ni_dv_ns_mv=DocValuesFormat(name=Asserting), number_p_f_dv_smf=DocValuesFormat(name=Lucene70), number_p_l_ni_dv_ns=DocValuesFormat(name=Lucene70), number_p_l_dv_smf=DocValuesFormat(name=Asserting), number_p_d_ni_mv_dv_sml=DocValuesFormat(name=Asserting), number_p_d_dv_smf=DocValuesFormat(name=Asserting), number_p_f_ni_dv_ns_mv=DocValuesFormat(name=Lucene70), number_p_dt_ni_dv=DocValuesFormat(name=Memory), number_p_f_ni_mv_dv_sml=DocValuesFormat(name=Lucene70), number_p_d_ni_mv_dv_smf=DocValuesFormat(name=Lucene70), number_p_f_dv_ns_mv=DocValuesFormat(name=Memory), number_p_l_dv_ns_mv=DocValuesFormat(name=Memory), number_p_dt_mv_dv_sml=DocValuesFormat(name=Lucene70), number_p_l_ni_ns_dv=DocValuesFormat(name=Memory), number_p_f_ni_ns_dv=DocValuesFormat(name=Lucene70), number_p_i_ni_dv_ns=DocValuesFormat(name=Lucene70), number_p_i_dv_sml=DocValuesFormat(name=Memory), foo_p_d_ni_dv_ns=DocValuesFormat(name=Lucene70), number_p_dt_dv_sml=DocValuesFormat(name=Asserting), number_p_dt_dv_smf=DocValuesFormat(name=Lucene70), number_p_d_ni_dv=DocValuesFormat(name=Lucene70), number_p_i_mv_dv=DocValuesFormat(name=Lucene70), number_p_l_ni_dv_ns_mv=DocValuesFormat(name=Memory), number_p_dt_ni_dv_sml=DocValuesFormat(name=Asserting), number_p_d_mv_dv_smf=DocValuesFormat(name=Lucene70), number_p_d_ni_dv_smf=DocValuesFormat(name=Asserting), number_p_dt_ni_dv_smf=DocValuesFormat(name=Lucene70), foo_p_l_ni_dv_ns_mv=DocValuesFormat(name=Asserting), number_p_d_mv_dv_sml=DocValuesFormat(name=Asserting), number_p_dt_ni_ns_dv=DocValuesFormat(name=Memory), foo_p_i_ni_dv_ns_mv=DocValuesFormat(name=Memory), number_p_dt_mv_dv_smf=DocValuesFormat(name=Asserting), number_p_d_ni_dv_sml=DocValuesFormat(name=Lucene70), number_p_l_mv_dv=DocValuesFormat(name=Lucene70), number_p_f_ni_mv_dv=DocValuesFormat(name=Lucene70), number_p_f_mv_dv=DocValuesFormat(name=Memory), number_p_i_ni_mv_dv=DocValuesFormat(name=Asserting), number_p_i_mv_dv_smf=DocValuesFormat(name=Lucene70), foo_p_dt_ni_dv_ns_mv=DocValuesFormat(name=Asserting), number_p_l_ni_mv_dv=DocValuesFormat(name=Memory), number_p_d_dv=DocValuesFormat(name=Lucene70), number_p_l_ni_dv_sml=DocValuesFormat(name=Lucene70), number_p_d_dv_ns=DocValuesFormat(name=Lucene70), number_p_i_ni_mv_dv_smf=DocValuesFormat(name=Lucene70), number_p_l_ni_dv_smf=DocValuesFormat(name=Asserting), number_p_i_ni_mv_dv_sml=DocValuesFormat(name=Memory), number_p_dt_dv_ns_mv=DocValuesFormat(name=Lucene70), number_p_l_dv=DocValuesFormat(name=Lucene70), number_p_dt_mv_dv=DocValuesFormat(name=Lucene70), number_p_l_mv_dv_smf=DocValuesFormat(name=Lucene70), number_p_d_ni_dv_ns=DocValuesFormat(name=Memory), number_p_i_dv_ns_mv=DocValuesFormat(name=Lucene70), number_p_d_ni_dv_ns_mv=DocValuesFormat(name=Lucene70), number_p_i_ni_dv_sml=DocValuesFormat(name=Lucene70), number_p_i_ni_dv_smf=DocValuesFormat(name=Memory), number_p_l_mv_dv_sml=DocValuesFormat(name=Asserting), number_p_f_dv_sml=DocValuesFormat(name=Asserting), number_p_d_dv_sml=DocValuesFormat(name=Lucene70), number_p_f_ni_dv_ns=DocValuesFormat(name=Memory), number_p_i_ni_ns_dv=DocValuesFormat(name=Asserting), number_p_f_ni_dv=DocValuesFormat(name=Lucene70), foo_p_dt_ni_dv_ns=DocValuesFormat(name=Lucene70), number_p_dt_dv=DocValuesFormat(name=Memory), number_p_l_ni_mv_dv_smf=DocValuesFormat(name=Lucene70), number_p_dt_ni_mv_dv_smf=DocValuesFormat(name=Asserting), number_p_d_ni_mv_dv=DocValuesFormat(name=Lucene70), number_p_d_dv_ns_mv=DocValuesFormat(name=Lucene70), number_p_f_ni_dv_sml=DocValuesFormat(name=Asserting), number_p_l_ni_mv_dv_sml=DocValuesFormat(name=Asserting), number_p_d_mv_dv=DocValuesFormat(name=Memory), number_p_i_dv=DocValuesFormat(name=Lucene70), number_p_f_mv_dv_sml=DocValuesFormat(name=Lucene70), number_p_i_mv_dv_sml=DocValuesFormat(name=Memory), number_p_f_ni_dv_smf=DocValuesFormat(name=Lucene70), number_p_dt_ni_dv_ns=DocValuesFormat(name=Lucene70), number_p_f_mv_dv_smf=DocValuesFormat(name=Asserting), number_p_i_ni_dv=DocValuesFormat(name=Asserting), number_p_f_dv_ns=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1696, maxMBSortInHeap=7.8315256312219255, sim=RandomSimilarity(queryNorm=false): {}, locale=ar, timezone=Europe/Kaliningrad
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=4,threads=1,free=60639968,total=534773760
   [junit4]   2> NOTE: All tests run in this JVM: [TestBulkSchemaAPI, TestCoreContainer, OverseerRolesTest, TestPolicyCloud, TestSolrJ, JsonLoaderTest, StressHdfsTest, TestConfigSetsAPIZkFailure, ChaosMonkeySafeLeaderTest, SmileWriterTest, TestTolerantUpdateProcessorCloud, TestRuleBasedAuthorizationPlugin, TestConfigSetImmutable, DocValuesMultiTest, ShardRoutingTest, TestBM25SimilarityFactory, TestCloudPseudoReturnFields, TestCollationFieldDocValues, TestFiltering, SoftAutoCommitTest, TestAtomicUpdateErrorCases, MetricsHandlerTest, TestSchemaManager, TestSolrIndexConfig, DistributedIntervalFacetingTest, DisMaxRequestHandlerTest, DistributedFacetPivotSmallTest, LeaderElectionContextKeyTest, TestGraphTermsQParserPlugin, CollectionsAPIDistributedZkTest, TestLegacyField, TestPullReplicaErrorHandling, CustomCollectionTest, MissingSegmentRecoveryTest, TestRecoveryHdfs, InfixSuggestersTest, ZkControllerTest, HdfsRestartWhileUpdatingTest, TestRawTransformer, BlockDirectoryTest, UninvertDocValuesMergePolicyTest, MetricUtilsTest, LoggingHandlerTest, DataDrivenBlockJoinTest, TestRTGBase, TestJsonRequest, TestRequestStatusCollectionAPI, AlternateDirectoryTest, TestFastLRUCache, MultiThreadedOCPTest, HLLSerializationTest, BadComponentTest, TestPivotHelperCode, ClusterStateUpdateTest, AnalysisAfterCoreReloadTest, ConfigSetsAPITest, TestPayloadCheckQParserPlugin, TestSolrConfigHandlerCloud, DirectSolrConnectionTest, DistributedMLTComponentTest, TestCorePropertiesReload, TestLRUStatsCache, TestManagedStopFilterFactory, TestLegacyTerms, TestMinMaxOnMultiValuedField, RankQueryTest, IndexSchemaTest, AddBlockUpdateTest, BitVectorTest, SolrXmlInZkTest, CdcrVersionReplicationTest, SubstringBytesRefFilterTest, DistribJoinFromCollectionTest, FileUtilsTest, TestSchemaNameResource, StatelessScriptUpdateProcessorFactoryTest, HdfsTlogReplayBufferedWhileIndexingTest, TestLRUCache, TermVectorComponentTest, TestMaxTokenLenTokenizer, TestStressRecovery, TestNRTOpen, SolrCmdDistributorTest, TestPostingsSolrHighlighter, CoreAdminHandlerTest, TestSimpleTrackingShardHandler, AsyncCallRequestStatusResponseTest, ZkCLITest, TestCloudInspectUtil, TestFieldCacheVsDocValues, CollectionReloadTest, TestDistributedGrouping, TestLeaderInitiatedRecoveryThread, HdfsChaosMonkeyNothingIsSafeTest, TestMiniSolrCloudCluster, UpdateRequestProcessorFactoryTest, SolrMetricManagerTest, DeleteNodeTest, EnumFieldTest, SliceStateTest, HighlighterTest, TestHdfsBackupRestoreCore, TestInPlaceUpdatesDistrib, TestImpersonationWithHadoopAuth, SuggesterTSTTest, SolrGangliaReporterTest, RollingRestartTest, TestRebalanceLeaders, TestCollapseQParserPlugin, TestMacroExpander, VersionInfoTest, PrimUtilsTest, TestSearchPerf, MoreLikeThisHandlerTest, TestSchemaVersionResource, ResourceLoaderTest, SystemInfoHandlerTest, RulesTest, DeleteLastCustomShardedReplicaTest, DeleteReplicaTest, DistributedQueueTest, DocValuesNotIndexedTest, ForceLeaderTest, LeaderInitiatedRecoveryOnCommitTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerCollectionConfigSetProcessorTest, PeerSyncReplicationTest, RemoteQueryErrorTest, ReplaceNodeTest, SharedFSAutoReplicaFailoverTest, SimpleCollectionCreateDeleteTest, TestCloudRecovery, TestClusterProperties, TestCollectionAPI, TestCryptoKeys, TestDeleteCollectionOnDownNodes, TestDistribDocBasedVersion, TestHdfsCloudBackupRestore, TestLeaderElectionWithEmptyReplica, TestMiniSolrCloudClusterSSL, TestRequestForwarding, TestSSLRandomization, TestSegmentSorting, TestStressLiveNodes, VMParamsZkACLAndCredentialsProvidersTest, AutoScalingHandlerTest, HdfsChaosMonkeySafeLeaderTest, HdfsNNFailoverTest, HdfsSyncSliceTest, ConfigureRecoveryStrategyTest, DirectoryFactoryTest, QueryResultKeyTest, TestBackupRepositoryFactory, TestConfigOverlay, TestConfigSetProperties, TestConfigSets, TestCustomStream, TestImplicitCoreProperties, TestReloadAndDeleteDocs, TestShardHandlerFactory, TestSimpleTextCodec, TestSolrCloudSnapshots, TestConfigReload, SegmentsInfoRequestHandlerTest, StatsReloadRaceTest, DistributedFacetPivotWhiteBoxTest, DistributedSuggestComponentTest, TestHttpShardHandlerFactory, HighlighterMaxOffsetTest, TestSlowCompositeReaderWrapper, CheckHdfsIndexTest, TestNumericRangeQuery64, SolrShardReporterTest, TestManagedSchemaAPI, TestPointFields]
   [junit4] Completed [659/726 (1!)] on J1 in 44.35s, 104 tests, 4 errors, 1 skipped <<< FAILURES!

[...truncated 43508 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-NightlyTests-7.0 - Build # 3 - Failure

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.0/3/

1 tests failed.
FAILED:  org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentAddAndDeleteAndCommit

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
        at __randomizedtesting.SeedInfo.seed([51B24AF99DC8185A:3DA7193FAC87B1A]:0)
        at java.lang.StringCoding.decode(StringCoding.java:215)
        at java.lang.String.<init>(String.java:463)
        at org.apache.lucene.store.DataInput.readString(DataInput.java:238)
        at org.apache.lucene.store.DataInput.readMapOfStrings(DataInput.java:273)
        at org.apache.lucene.codecs.lucene70.Lucene70SegmentInfoFormat.read(Lucene70SegmentInfoFormat.java:119)
        at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:355)
        at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:289)
        at org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:277)
        at org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentAddAndDeleteAndCommit(TestIndexingSequenceNumbers.java:554)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)




Build Log:
[...truncated 851 lines...]
   [junit4] Suite: org.apache.lucene.index.TestIndexingSequenceNumbers
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexingSequenceNumbers -Dtests.method=testStressConcurrentAddAndDeleteAndCommit -Dtests.seed=51B24AF99DC8185A -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/test-data/enwiki.random.lines.txt -Dtests.locale=id -Dtests.timezone=America/Grenada -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    586s J0 | TestIndexingSequenceNumbers.testStressConcurrentAddAndDeleteAndCommit <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > at __randomizedtesting.SeedInfo.seed([51B24AF99DC8185A:3DA7193FAC87B1A]:0)
   [junit4]    > at java.lang.StringCoding.decode(StringCoding.java:215)
   [junit4]    > at java.lang.String.<init>(String.java:463)
   [junit4]    > at org.apache.lucene.store.DataInput.readString(DataInput.java:238)
   [junit4]    > at org.apache.lucene.store.DataInput.readMapOfStrings(DataInput.java:273)
   [junit4]    > at org.apache.lucene.codecs.lucene70.Lucene70SegmentInfoFormat.read(Lucene70SegmentInfoFormat.java:119)
   [junit4]    > at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:355)
   [junit4]    > at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:289)
   [junit4]    > at org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:277)
   [junit4]    > at org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentAddAndDeleteAndCommit(TestIndexingSequenceNumbers.java:554)
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/lucene/build/core/test/J0/temp/lucene.index.TestIndexingSequenceNumbers_51B24AF99DC8185A-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=MockRandom)}, docValues:{thread=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1138, maxMBSortInHeap=7.90637858347033, sim=RandomSimilarity(queryNorm=true): {id=DFR I(F)BZ(0.3)}, locale=id, timezone=America/Grenada
   [junit4]   2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=4,threads=1,free=117857288,total=505937920
   [junit4]   2> NOTE: All tests run in this JVM: [TestTotalHitCountCollector, TestIndexedDISI, TestReaderWrapperDVTypeCheck, TestLongBitSet, TestNewestSegment, TestNGramPhraseQuery, TestIntRangeFieldQueries, TestRegexpRandom, TestPerFieldPostingsFormat, TestDirectoryReaderReopen, TestNot, TestIndexableField, TestSynonymQuery, TestUnicodeUtil, TestExitableDirectoryReader, TestSpansEnum, TestClassicSimilarity, TestAxiomaticSimilarity, TestStandardAnalyzer, TestGeoUtils, TestSearch, Test4GBStoredFields, TestPostingsOffsets, TestDocValuesIndexing, TestIndexWriterConfig, TestCodecHoldsOpenFiles, TestSimpleAttributeImpl, TestPerSegmentDeletes, TestIntBlockPool, TestTopFieldCollector, TestEarlyTermination, TestSpanCollection, TestNeverDelete, TestAllFilesCheckIndexHeader, TestMultiThreadTermVectors, TestWordlistLoader, TestIndexWriterExceptions2, TestDocsAndPositions, TestSimpleSearchEquivalence, TestDateTools, TestIndexingSequenceNumbers]
   [junit4] Completed [200/454 (1!)] on J0 in 659.08s, 8 tests, 1 error <<< FAILURES!

[...truncated 865 lines...]
   [junit4] JVM J0: stdout was not empty, see: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/lucene/build/core/test/temp/junit4-J0-20170712_202417_2631435964930006081507.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/heapdumps/java_pid5240.hprof ...
   [junit4] Heap dump file created [563001186 bytes in 2.359 secs]
   [junit4] <<< JVM J0: EOF ----

[...truncated 19433 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/build.xml:818: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/build.xml:762: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid5240.hprof

Total time: 217 minutes 0 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-NightlyTests-7.0 - Build # 4 - Still Failing

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.0/4/

3 tests failed.
FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
Didn't see all replicas for shard shard1 in c8n_1x2 come up within 90000 ms! ClusterState: {   "collMinRf_1x3":{     "pullReplicas":"0",     "replicationFactor":"1",     "shards":{"shard1":{         "range":"80000000-7fffffff",         "state":"active",         "replicas":{           "core_node1":{             "core":"collMinRf_1x3_shard1_replica_t1",             "base_url":"http://127.0.0.1:33851/bz_/a",             "node_name":"127.0.0.1:33851_bz_%2Fa",             "state":"active",             "type":"TLOG"},           "core_node2":{             "core":"collMinRf_1x3_shard1_replica_t2",             "base_url":"http://127.0.0.1:41145/bz_/a",             "node_name":"127.0.0.1:41145_bz_%2Fa",             "state":"active",             "type":"TLOG",             "leader":"true"},           "core_node3":{             "core":"collMinRf_1x3_shard1_replica_t3",             "base_url":"http://127.0.0.1:40101/bz_/a",             "node_name":"127.0.0.1:40101_bz_%2Fa",             "state":"active",             "type":"TLOG"}}}},     "router":{"name":"compositeId"},     "maxShardsPerNode":"1",     "autoAddReplicas":"false",     "nrtReplicas":"0",     "tlogReplicas":"3"},   "collection1":{     "pullReplicas":"0",     "replicationFactor":"1",     "shards":{       "shard1":{         "range":"80000000-ffffffff",         "state":"active",         "replicas":{"core_node2":{             "core":"collection1_shard1_replica_t0",             "base_url":"http://127.0.0.1:40101/bz_/a",             "node_name":"127.0.0.1:40101_bz_%2Fa",             "state":"active",             "type":"TLOG",             "leader":"true"}}},       "shard2":{         "range":"0-7fffffff",         "state":"active",         "replicas":{           "core_node1":{             "core":"collection1_shard2_replica_t0",             "base_url":"http://127.0.0.1:33851/bz_/a",             "node_name":"127.0.0.1:33851_bz_%2Fa",             "state":"active",             "type":"TLOG",             "leader":"true"},           "core_node3":{             "core":"collection1_shard2_replica_t1",             "base_url":"http://127.0.0.1:38095/bz_/a",             "node_name":"127.0.0.1:38095_bz_%2Fa",             "state":"active",             "type":"TLOG"}}}},     "router":{"name":"compositeId"},     "maxShardsPerNode":"1",     "autoAddReplicas":"false",     "nrtReplicas":"1",     "tlogReplicas":"0"},   "control_collection":{     "pullReplicas":"0",     "replicationFactor":"1",     "shards":{"shard1":{         "range":"80000000-7fffffff",         "state":"active",         "replicas":{"core_node1":{             "core":"control_collection_shard1_replica_n1",             "base_url":"http://127.0.0.1:41145/bz_/a",             "node_name":"127.0.0.1:41145_bz_%2Fa",             "state":"active",             "type":"NRT",             "leader":"true"}}}},     "router":{"name":"compositeId"},     "maxShardsPerNode":"1",     "autoAddReplicas":"false",     "nrtReplicas":"1",     "tlogReplicas":"0"},   "c8n_1x2":{     "pullReplicas":"0",     "replicationFactor":"1",     "shards":{"shard1":{         "range":"80000000-7fffffff",         "state":"active",         "replicas":{           "core_node1":{             "core":"c8n_1x2_shard1_replica_t1",             "base_url":"http://127.0.0.1:40101/bz_/a",             "node_name":"127.0.0.1:40101_bz_%2Fa",             "state":"active",             "type":"TLOG",             "leader":"true"},           "core_node2":{             "state":"down",             "base_url":"http://127.0.0.1:41145/bz_/a",             "core":"c8n_1x2_shard1_replica_t2",             "node_name":"127.0.0.1:41145_bz_%2Fa",             "type":"TLOG"}}}},     "router":{"name":"compositeId"},     "maxShardsPerNode":"1",     "autoAddReplicas":"false",     "nrtReplicas":"0",     "tlogReplicas":"2"}}

Stack Trace:
java.lang.AssertionError: Didn't see all replicas for shard shard1 in c8n_1x2 come up within 90000 ms! ClusterState: {
  "collMinRf_1x3":{
    "pullReplicas":"0",
    "replicationFactor":"1",
    "shards":{"shard1":{
        "range":"80000000-7fffffff",
        "state":"active",
        "replicas":{
          "core_node1":{
            "core":"collMinRf_1x3_shard1_replica_t1",
            "base_url":"http://127.0.0.1:33851/bz_/a",
            "node_name":"127.0.0.1:33851_bz_%2Fa",
            "state":"active",
            "type":"TLOG"},
          "core_node2":{
            "core":"collMinRf_1x3_shard1_replica_t2",
            "base_url":"http://127.0.0.1:41145/bz_/a",
            "node_name":"127.0.0.1:41145_bz_%2Fa",
            "state":"active",
            "type":"TLOG",
            "leader":"true"},
          "core_node3":{
            "core":"collMinRf_1x3_shard1_replica_t3",
            "base_url":"http://127.0.0.1:40101/bz_/a",
            "node_name":"127.0.0.1:40101_bz_%2Fa",
            "state":"active",
            "type":"TLOG"}}}},
    "router":{"name":"compositeId"},
    "maxShardsPerNode":"1",
    "autoAddReplicas":"false",
    "nrtReplicas":"0",
    "tlogReplicas":"3"},
  "collection1":{
    "pullReplicas":"0",
    "replicationFactor":"1",
    "shards":{
      "shard1":{
        "range":"80000000-ffffffff",
        "state":"active",
        "replicas":{"core_node2":{
            "core":"collection1_shard1_replica_t0",
            "base_url":"http://127.0.0.1:40101/bz_/a",
            "node_name":"127.0.0.1:40101_bz_%2Fa",
            "state":"active",
            "type":"TLOG",
            "leader":"true"}}},
      "shard2":{
        "range":"0-7fffffff",
        "state":"active",
        "replicas":{
          "core_node1":{
            "core":"collection1_shard2_replica_t0",
            "base_url":"http://127.0.0.1:33851/bz_/a",
            "node_name":"127.0.0.1:33851_bz_%2Fa",
            "state":"active",
            "type":"TLOG",
            "leader":"true"},
          "core_node3":{
            "core":"collection1_shard2_replica_t1",
            "base_url":"http://127.0.0.1:38095/bz_/a",
            "node_name":"127.0.0.1:38095_bz_%2Fa",
            "state":"active",
            "type":"TLOG"}}}},
    "router":{"name":"compositeId"},
    "maxShardsPerNode":"1",
    "autoAddReplicas":"false",
    "nrtReplicas":"1",
    "tlogReplicas":"0"},
  "control_collection":{
    "pullReplicas":"0",
    "replicationFactor":"1",
    "shards":{"shard1":{
        "range":"80000000-7fffffff",
        "state":"active",
        "replicas":{"core_node1":{
            "core":"control_collection_shard1_replica_n1",
            "base_url":"http://127.0.0.1:41145/bz_/a",
            "node_name":"127.0.0.1:41145_bz_%2Fa",
            "state":"active",
            "type":"NRT",
            "leader":"true"}}}},
    "router":{"name":"compositeId"},
    "maxShardsPerNode":"1",
    "autoAddReplicas":"false",
    "nrtReplicas":"1",
    "tlogReplicas":"0"},
  "c8n_1x2":{
    "pullReplicas":"0",
    "replicationFactor":"1",
    "shards":{"shard1":{
        "range":"80000000-7fffffff",
        "state":"active",
        "replicas":{
          "core_node1":{
            "core":"c8n_1x2_shard1_replica_t1",
            "base_url":"http://127.0.0.1:40101/bz_/a",
            "node_name":"127.0.0.1:40101_bz_%2Fa",
            "state":"active",
            "type":"TLOG",
            "leader":"true"},
          "core_node2":{
            "state":"down",
            "base_url":"http://127.0.0.1:41145/bz_/a",
            "core":"c8n_1x2_shard1_replica_t2",
            "node_name":"127.0.0.1:41145_bz_%2Fa",
            "type":"TLOG"}}}},
    "router":{"name":"compositeId"},
    "maxShardsPerNode":"1",
    "autoAddReplicas":"false",
    "nrtReplicas":"0",
    "tlogReplicas":"2"}}
        at __randomizedtesting.SeedInfo.seed([7E60B1F944FBDF16:F6348E23EA07B2EE]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.ensureAllReplicasAreActive(AbstractFullDistribZkTestBase.java:1992)
        at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:367)
        at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:131)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test

Error Message:
Timeout waiting for all live and active

Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
        at __randomizedtesting.SeedInfo.seed([7E60B1F944FBDF16:F6348E23EA07B2EE]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:317)
        at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Didn't get expected doc count. Expected: 10, Found: 0

Stack Trace:
java.lang.AssertionError: Didn't get expected doc count. Expected: 10, Found: 0
        at __randomizedtesting.SeedInfo.seed([7E60B1F944FBDF16:F06CD18931E35B3C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.assertNumDocs(TestPullReplicaErrorHandling.java:245)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.assertNumDocs(TestPullReplicaErrorHandling.java:250)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader(TestPullReplicaErrorHandling.java:189)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 11882 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestPullReplicaErrorHandling
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_7E60B1F944FBDF16-001/init-core-data-001
   [junit4]   2> 1329836 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1329836 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 1329837 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_7E60B1F944FBDF16-001/tempDir-001
   [junit4]   2> 1329837 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1329837 INFO  (Thread-3622) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1329837 INFO  (Thread-3622) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1329854 ERROR (Thread-3622) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1329937 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:33349
   [junit4]   2> 1329942 INFO  (jetty-launcher-365-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1329944 INFO  (jetty-launcher-365-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5a1e15{/solr,null,AVAILABLE}
   [junit4]   2> 1329944 INFO  (jetty-launcher-365-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1329945 INFO  (jetty-launcher-365-thread-3) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1329945 INFO  (jetty-launcher-365-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5304c3da{HTTP/1.1,[http/1.1]}{127.0.0.1:36772}
   [junit4]   2> 1329945 INFO  (jetty-launcher-365-thread-1) [    ] o.e.j.s.Server Started @1340528ms
   [junit4]   2> 1329945 INFO  (jetty-launcher-365-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36772}
   [junit4]   2> 1329946 ERROR (jetty-launcher-365-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1329946 INFO  (jetty-launcher-365-thread-4) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1329946 INFO  (jetty-launcher-365-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 1329946 INFO  (jetty-launcher-365-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1329946 INFO  (jetty-launcher-365-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1329946 INFO  (jetty-launcher-365-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-13T23:26:08.187Z
   [junit4]   2> 1329960 INFO  (jetty-launcher-365-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@54c501fb{/solr,null,AVAILABLE}
   [junit4]   2> 1329960 INFO  (jetty-launcher-365-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@268f4640{/solr,null,AVAILABLE}
   [junit4]   2> 1329960 INFO  (jetty-launcher-365-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62c6727c{/solr,null,AVAILABLE}
   [junit4]   2> 1329960 INFO  (jetty-launcher-365-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1329960 INFO  (jetty-launcher-365-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@71e907cc{HTTP/1.1,[http/1.1]}{127.0.0.1:41470}
   [junit4]   2> 1329960 INFO  (jetty-launcher-365-thread-3) [    ] o.e.j.s.Server Started @1340543ms
   [junit4]   2> 1329961 INFO  (jetty-launcher-365-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41470}
   [junit4]   2> 1329965 INFO  (jetty-launcher-365-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1f410eaa{HTTP/1.1,[http/1.1]}{127.0.0.1:42440}
   [junit4]   2> 1329965 INFO  (jetty-launcher-365-thread-2) [    ] o.e.j.s.Server Started @1340548ms
   [junit4]   2> 1329965 INFO  (jetty-launcher-365-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42440}
   [junit4]   2> 1329961 INFO  (jetty-launcher-365-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4ae404b3{HTTP/1.1,[http/1.1]}{127.0.0.1:37623}
   [junit4]   2> 1329965 INFO  (jetty-launcher-365-thread-4) [    ] o.e.j.s.Server Started @1340548ms
   [junit4]   2> 1329965 ERROR (jetty-launcher-365-thread-3) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1329965 ERROR (jetty-launcher-365-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1329966 INFO  (jetty-launcher-365-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 1329965 INFO  (jetty-launcher-365-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37623}
   [junit4]   2> 1329966 INFO  (jetty-launcher-365-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 1329966 INFO  (jetty-launcher-365-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1329966 INFO  (jetty-launcher-365-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1329966 INFO  (jetty-launcher-365-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1329966 INFO  (jetty-launcher-365-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1329966 INFO  (jetty-launcher-365-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-13T23:26:08.207Z
   [junit4]   2> 1329966 INFO  (jetty-launcher-365-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-13T23:26:08.207Z
   [junit4]   2> 1329966 ERROR (jetty-launcher-365-thread-4) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1329966 INFO  (jetty-launcher-365-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 1329966 INFO  (jetty-launcher-365-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1329966 INFO  (jetty-launcher-365-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1329966 INFO  (jetty-launcher-365-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-13T23:26:08.207Z
   [junit4]   2> 1329977 INFO  (jetty-launcher-365-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1329978 INFO  (jetty-launcher-365-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33349/solr
   [junit4]   2> 1329996 INFO  (jetty-launcher-365-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1329997 INFO  (jetty-launcher-365-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1330000 INFO  (jetty-launcher-365-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33349/solr
   [junit4]   2> 1330039 INFO  (jetty-launcher-365-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33349/solr
   [junit4]   2> 1330053 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15d3e461ade0006, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1330057 INFO  (jetty-launcher-365-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33349/solr
   [junit4]   2> 1330098 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15d3e461ade000a, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1330279 INFO  (jetty-launcher-365-thread-4) [n:127.0.0.1:37623_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1330280 INFO  (jetty-launcher-365-thread-4) [n:127.0.0.1:37623_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37623_solr
   [junit4]   2> 1330293 INFO  (jetty-launcher-365-thread-4) [n:127.0.0.1:37623_solr    ] o.a.s.c.Overseer Overseer (id=98303237691408396-127.0.0.1:37623_solr-n_0000000000) starting
   [junit4]   2> 1330325 INFO  (jetty-launcher-365-thread-4) [n:127.0.0.1:37623_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37623_solr
   [junit4]   2> 1330339 INFO  (zkCallback-387-thread-1-processing-n:127.0.0.1:37623_solr) [n:127.0.0.1:37623_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1330506 INFO  (jetty-launcher-365-thread-1) [n:127.0.0.1:36772_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1330521 INFO  (jetty-launcher-365-thread-1) [n:127.0.0.1:36772_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1330522 INFO  (jetty-launcher-365-thread-1) [n:127.0.0.1:36772_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36772_solr
   [junit4]   2> 1330540 INFO  (zkCallback-387-thread-2-processing-n:127.0.0.1:37623_solr) [n:127.0.0.1:37623_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1330621 INFO  (zkCallback-378-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1330728 INFO  (jetty-launcher-365-thread-4) [n:127.0.0.1:37623_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37623.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1330744 INFO  (jetty-launcher-365-thread-4) [n:127.0.0.1:37623_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37623.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1330744 INFO  (jetty-launcher-365-thread-4) [n:127.0.0.1:37623_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37623.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1330748 INFO  (jetty-launcher-365-thread-4) [n:127.0.0.1:37623_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_7E60B1F944FBDF16-001/tempDir-001/node4/.
   [junit4]   2> 1330751 INFO  (jetty-launcher-365-thread-2) [n:127.0.0.1:42440_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1330765 INFO  (jetty-launcher-365-thread-2) [n:127.0.0.1:42440_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1330768 INFO  (jetty-launcher-365-thread-2) [n:127.0.0.1:42440_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42440_solr
   [junit4]   2> 1330813 INFO  (jetty-launcher-365-thread-1) [n:127.0.0.1:36772_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36772.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1330816 INFO  (jetty-launcher-365-thread-1) [n:127.0.0.1:36772_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36772.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1330817 INFO  (jetty-launcher-365-thread-1) [n:127.0.0.1:36772_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36772.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1330829 INFO  (zkCallback-378-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1330916 INFO  (zkCallback-386-thread-1-processing-n:127.0.0.1:42440_solr) [n:127.0.0.1:42440_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1330926 INFO  (zkCallback-387-thread-2-processing-n:127.0.0.1:37623_solr) [n:127.0.0.1:37623_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1330929 INFO  (jetty-launcher-365-thread-1) [n:127.0.0.1:36772_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_7E60B1F944FBDF16-001/tempDir-001/node1/.
   [junit4]   2> 1330991 INFO  (jetty-launcher-365-thread-3) [n:127.0.0.1:41470_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1331021 INFO  (jetty-launcher-365-thread-3) [n:127.0.0.1:41470_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1331059 INFO  (jetty-launcher-365-thread-3) [n:127.0.0.1:41470_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41470_solr
   [junit4]   2> 1331060 INFO  (zkCallback-386-thread-1-processing-n:127.0.0.1:42440_solr) [n:127.0.0.1:42440_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1331060 INFO  (zkCallback-387-thread-2-processing-n:127.0.0.1:37623_solr) [n:127.0.0.1:37623_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1331108 INFO  (zkCallback-383-thread-1-processing-n:127.0.0.1:41470_solr) [n:127.0.0.1:41470_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1331179 INFO  (zkCallback-378-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1331228 INFO  (jetty-launcher-365-thread-2) [n:127.0.0.1:42440_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42440.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1331260 INFO  (jetty-launcher-365-thread-2) [n:127.0.0.1:42440_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42440.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1331260 INFO  (jetty-launcher-365-thread-2) [n:127.0.0.1:42440_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42440.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1331261 INFO  (jetty-launcher-365-thread-2) [n:127.0.0.1:42440_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_7E60B1F944FBDF16-001/tempDir-001/node2/.
   [junit4]   2> 1331566 INFO  (jetty-launcher-365-thread-3) [n:127.0.0.1:41470_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41470.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1331569 INFO  (jetty-launcher-365-thread-3) [n:127.0.0.1:41470_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41470.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1331569 INFO  (jetty-launcher-365-thread-3) [n:127.0.0.1:41470_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41470.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1331570 INFO  (jetty-launcher-365-thread-3) [n:127.0.0.1:41470_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_7E60B1F944FBDF16-001/tempDir-001/node3/.
   [junit4]   2> 1331676 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1331677 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33349/solr ready
   [junit4]   2> 1331680 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@4ae404b3{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1331681 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2071021257
   [junit4]   2> 1331681 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1331682 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1331683 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1331684 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1331699 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1331699 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.Overseer Overseer (id=98303237691408396-127.0.0.1:37623_solr-n_0000000000) closing
   [junit4]   2> 1331699 INFO  (OverseerStateUpdate-98303237691408396-127.0.0.1:37623_solr-n_0000000000) [n:127.0.0.1:37623_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37623_solr
   [junit4]   2> 1331701 INFO  (zkCallback-378-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36772_solr
   [junit4]   2> 1331701 INFO  (zkCallback-386-thread-1-processing-n:127.0.0.1:42440_solr) [n:127.0.0.1:42440_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1331701 INFO  (zkCallback-383-thread-1-processing-n:127.0.0.1:41470_solr) [n:127.0.0.1:41470_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1331702 INFO  (zkCallback-378-thread-2-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1331702 INFO  (zkCallback-395-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1331702 INFO  (zkCallback-378-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr    ] o.a.s.c.Overseer Overseer (id=98303237691408391-127.0.0.1:36772_solr-n_0000000001) starting
   [junit4]   2> 1333201 WARN  (zkCallback-387-thread-2-processing-n:127.0.0.1:37623_solr) [n:127.0.0.1:37623_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1333202 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@268f4640{/solr,null,UNAVAILABLE}
   [junit4]   2> 1333203 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1333204 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4118a72e{/solr,null,AVAILABLE}
   [junit4]   2> 1333204 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7fe00e23{HTTP/1.1,[http/1.1]}{127.0.0.1:41500}
   [junit4]   2> 1333204 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.Server Started @1343787ms
   [junit4]   2> 1333204 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42185}
   [junit4]   2> 1333205 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1333205 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 1333205 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1333205 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1333205 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-13T23:26:11.446Z
   [junit4]   2> 1333215 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1333260 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33349/solr
   [junit4]   2> 1333286 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:42185_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1333288 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:42185_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1333289 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:42185_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42185_solr
   [junit4]   2> 1333290 INFO  (zkCallback-383-thread-1-processing-n:127.0.0.1:41470_solr) [n:127.0.0.1:41470_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1333290 INFO  (zkCallback-395-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1333290 INFO  (zkCallback-386-thread-1-processing-n:127.0.0.1:42440_solr) [n:127.0.0.1:42440_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1333290 INFO  (zkCallback-378-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1333291 INFO  (zkCallback-400-thread-1-processing-n:127.0.0.1:42185_solr) [n:127.0.0.1:42185_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1333511 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:42185_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42185.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1333523 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:42185_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42185.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1333523 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:42185_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42185.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1333524 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:42185_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_7E60B1F944FBDF16-001/tempDir-001/node4/.
   [junit4]   2> 1333581 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:41500/solr. Proxy: http://127.0.0.1:42185/solr
   [junit4]   2> 1333582 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@5304c3da{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1333582 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=628433303
   [junit4]   2> 1333583 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1333584 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1333585 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1333586 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1333587 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1333587 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.Overseer Overseer (id=98303237691408391-127.0.0.1:36772_solr-n_0000000001) closing
   [junit4]   2> 1333587 INFO  (OverseerStateUpdate-98303237691408391-127.0.0.1:36772_solr-n_0000000001) [n:127.0.0.1:36772_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36772_solr
   [junit4]   2> 1333599 INFO  (zkCallback-395-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1333599 INFO  (zkCallback-400-thread-1-processing-n:127.0.0.1:42185_solr) [n:127.0.0.1:42185_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1333599 INFO  (zkCallback-383-thread-1-processing-n:127.0.0.1:41470_solr) [n:127.0.0.1:41470_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1333599 INFO  (zkCallback-386-thread-1-processing-n:127.0.0.1:42440_solr) [n:127.0.0.1:42440_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42440_solr
   [junit4]   2> 1333599 INFO  (zkCallback-386-thread-2-processing-n:127.0.0.1:42440_solr) [n:127.0.0.1:42440_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1333599 INFO  (zkCallback-386-thread-1-processing-n:127.0.0.1:42440_solr) [n:127.0.0.1:42440_solr    ] o.a.s.c.Overseer Overseer (id=98303237691408395-127.0.0.1:42440_solr-n_0000000002) starting
   [junit4]   2> 1335099 WARN  (zkCallback-378-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1335099 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5a1e15{/solr,null,UNAVAILABLE}
   [junit4]   2> 1335100 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1335101 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@21488413{/solr,null,AVAILABLE}
   [junit4]   2> 1335101 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2108ce6c{HTTP/1.1,[http/1.1]}{127.0.0.1:41510}
   [junit4]   2> 1335101 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.Server Started @1345684ms
   [junit4]   2> 1335102 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40509}
   [junit4]   2> 1335102 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1335102 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 1335102 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1335102 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1335102 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-13T23:26:13.343Z
   [junit4]   2> 1335104 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1335111 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33349/solr
   [junit4]   2> 1335122 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:40509_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1335136 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:40509_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1335137 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:40509_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40509_solr
   [junit4]   2> 1335138 INFO  (zkCallback-383-thread-1-processing-n:127.0.0.1:41470_solr) [n:127.0.0.1:41470_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1335138 INFO  (zkCallback-400-thread-1-processing-n:127.0.0.1:42185_solr) [n:127.0.0.1:42185_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1335138 INFO  (zkCallback-395-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1335153 INFO  (zkCallback-386-thread-2-processing-n:127.0.0.1:42440_solr) [n:127.0.0.1:42440_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1335153 INFO  (zkCallback-406-thread-1-processing-n:127.0.0.1:40509_solr) [n:127.0.0.1:40509_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1335404 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:40509_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40509.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1335408 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:40509_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40509.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1335408 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:40509_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40509.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1335409 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:40509_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_7E60B1F944FBDF16-001/tempDir-001/node1/.
   [junit4]   2> 1335433 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:41510/solr. Proxy: http://127.0.0.1:40509/solr
   [junit4]   2> 1335433 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@1f410eaa{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1335434 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=190917814
   [junit4]   2> 1335434 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1335436 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1335445 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1335446 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1335446 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1335447 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.Overseer Overseer (id=98303237691408395-127.0.0.1:42440_solr-n_0000000002) closing
   [junit4]   2> 1335447 INFO  (OverseerStateUpdate-98303237691408395-127.0.0.1:42440_solr-n_0000000002) [n:127.0.0.1:42440_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42440_solr
   [junit4]   2> 1335469 INFO  (zkCallback-383-thread-1-processing-n:127.0.0.1:41470_solr) [n:127.0.0.1:41470_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1335469 INFO  (zkCallback-400-thread-1-processing-n:127.0.0.1:42185_solr) [n:127.0.0.1:42185_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1335469 INFO  (zkCallback-406-thread-1-processing-n:127.0.0.1:40509_solr) [n:127.0.0.1:40509_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1335469 INFO  (zkCallback-395-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1335469 INFO  (zkCallback-383-thread-2-processing-n:127.0.0.1:41470_solr) [n:127.0.0.1:41470_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41470_solr
   [junit4]   2> 1335470 INFO  (zkCallback-383-thread-2-processing-n:127.0.0.1:41470_solr) [n:127.0.0.1:41470_solr    ] o.a.s.c.Overseer Overseer (id=98303237691408393-127.0.0.1:41470_solr-n_0000000003) starting
   [junit4]   2> 1336970 WARN  (zkCallback-386-thread-2-processing-n:127.0.0.1:42440_solr) [n:127.0.0.1:42440_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1336970 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@54c501fb{/solr,null,UNAVAILABLE}
   [junit4]   2> 1336986 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1336987 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@15c59d80{/solr,null,AVAILABLE}
   [junit4]   2> 1336987 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@676f3a4c{HTTP/1.1,[http/1.1]}{127.0.0.1:34409}
   [junit4]   2> 1336987 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.Server Started @1347570ms
   [junit4]   2> 1336988 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44462}
   [junit4]   2> 1336988 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1336989 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 1336989 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1336989 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1336989 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-13T23:26:15.230Z
   [junit4]   2> 1337014 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1337059 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33349/solr
   [junit4]   2> 1337148 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:44462_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1337163 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:44462_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1337164 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:44462_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44462_solr
   [junit4]   2> 1337184 INFO  (zkCallback-406-thread-1-processing-n:127.0.0.1:40509_solr) [n:127.0.0.1:40509_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1337214 INFO  (zkCallback-412-thread-1-processing-n:127.0.0.1:44462_solr) [n:127.0.0.1:44462_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1337230 INFO  (zkCallback-400-thread-1-processing-n:127.0.0.1:42185_solr) [n:127.0.0.1:42185_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1337230 INFO  (zkCallback-395-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1337264 INFO  (zkCallback-383-thread-1-processing-n:127.0.0.1:41470_solr) [n:127.0.0.1:41470_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1337565 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:44462_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44462.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1337581 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:44462_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44462.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1337581 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:44462_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44462.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1337582 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:44462_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_7E60B1F944FBDF16-001/tempDir-001/node2/.
   [junit4]   2> 1337693 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:34409/solr. Proxy: http://127.0.0.1:44462/solr
   [junit4]   2> 1337695 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@71e907cc{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1337695 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=640344857
   [junit4]   2> 1337695 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1337697 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1337698 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1337701 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1337713 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1337713 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.Overseer Overseer (id=98303237691408393-127.0.0.1:41470_solr-n_0000000003) closing
   [junit4]   2> 1337735 INFO  (OverseerStateUpdate-98303237691408393-127.0.0.1:41470_solr-n_0000000003) [n:127.0.0.1:41470_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:41470_solr
   [junit4]   2> 1337746 INFO  (zkCallback-406-thread-1-processing-n:127.0.0.1:40509_solr) [n:127.0.0.1:40509_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1337747 INFO  (zkCallback-395-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1337756 INFO  (zkCallback-412-thread-1-processing-n:127.0.0.1:44462_solr) [n:127.0.0.1:44462_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1337756 INFO  (zkCallback-400-thread-1-processing-n:127.0.0.1:42185_solr) [n:127.0.0.1:42185_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42185_solr
   [junit4]   2> 1337757 INFO  (zkCallback-400-thread-1-processing-n:127.0.0.1:42185_solr) [n:127.0.0.1:42185_solr    ] o.a.s.c.Overseer Overseer (id=98303237691408401-127.0.0.1:42185_solr-n_0000000004) starting
   [junit4]   2> 1337758 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@62c6727c{/solr,null,UNAVAILABLE}
   [junit4]   2> 1337759 INFO  (zkCallback-400-thread-2-processing-n:127.0.0.1:42185_solr) [n:127.0.0.1:42185_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1337811 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1337852 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6f9d8983{/solr,null,AVAILABLE}
   [junit4]   2> 1337852 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@e28cd04{HTTP/1.1,[http/1.1]}{127.0.0.1:44078}
   [junit4]   2> 1337852 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.e.j.s.Server Started @1348435ms
   [junit4]   2> 1337860 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36469}
   [junit4]   2> 1337860 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1337860 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.0.0
   [junit4]   2> 1337860 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1337860 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1337860 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-13T23:26:16.101Z
   [junit4]   2> 1337891 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1337975 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33349/solr
   [junit4]   2> 1338100 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:36469_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1338102 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:36469_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1338158 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:36469_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36469_solr
   [junit4]   2> 1338239 INFO  (zkCallback-418-thread-1-processing-n:127.0.0.1:36469_solr) [n:127.0.0.1:36469_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1338252 INFO  (zkCallback-395-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1338256 INFO  (zkCallback-406-thread-1-processing-n:127.0.0.1:40509_solr) [n:127.0.0.1:40509_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1338286 INFO  (zkCallback-412-thread-1-processing-n:127.0.0.1:44462_solr) [n:127.0.0.1:44462_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1338298 INFO  (zkCallback-400-thread-1-processing-n:127.0.0.1:42185_solr) [n:127.0.0.1:42185_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1338796 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:36469_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36469.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1339135 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:36469_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36469.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1339135 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:36469_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36469.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1339136 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [n:127.0.0.1:36469_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_7E60B1F944FBDF16-001/tempDir-001/node3/.
   [junit4]   2> 1339342 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[7E60B1F944FBDF16]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:44078/solr. Proxy: http://127.0.0.1:36469/solr
   [junit4]   2> 1339347 INFO  (SocketProxy-Acceptor-36469) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44490,localport=36469], receiveBufferSize:531000
   [junit4]   2> 1339350 INFO  (SocketProxy-Acceptor-36469) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=44078,localport=57642], receiveBufferSize=530904
   [junit4]   2> 1339380 INFO  (qtp394965859-6447) [n:127.0.0.1:36469_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1339384 INFO  (qtp394965859-6447) [n:127.0.0.1:36469_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 1339467 INFO  (TEST-TestPullReplicaErrorHandling.testCantConnectToPullReplica-seed#[7E60B1F944FBDF16]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCantConnectToPullReplica
   [junit4]   2> 1339492 INFO  (qtp394965859-6448) [n:127.0.0.1:36469_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=1&replicationFactor=1&collection.configName=conf&maxShardsPerNode=1&name=pull_replica_error_handling_test_cant_connect_to_pull_replica&nrtReplicas=1&action=CREATE&numShards=2&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1339541 INFO  (OverseerThreadFactory-1038-thread-1-processing-n:127.0.0.1:42185_solr) [n:127.0.0.1:42185_solr    ] o.a.s.c.CreateCollectionCmd Create collection pull_replica_error_handling_test_cant_connect_to_pull_replica
   [junit4]   2> 1339774 INFO  (OverseerStateUpdate-98303237691408401-127.0.0.1:42185_solr-n_0000000004) [n:127.0.0.1:42185_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"pull_replica_error_handling_test_cant_connect_to_pull_replica",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:44462/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1339776 INFO  (OverseerStateUpdate-98303237691408401-127.0.0.1:42185_solr-n_0000000004) [n:127.0.0.1:42185_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"pull_replica_error_handling_test_cant_connect_to_pull_replica",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_p1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:36469/solr",
   [junit4]   2>   "type":"PULL"}
   [junit4]   2> 1339802 INFO  (OverseerStateUpdate-98303237691408401-127.0.0.1:42185_solr-n_0000000004) [n:127.0.0.1:42185_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"pull_replica_error_handling_test_cant_connect_to_pull_replica",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"pull_replica_error_handling_test_cant_connect_to_pull_replica_shard2_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:40509/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1339818 INFO  (OverseerStateUpdate-98303237691408401-127.0.0.1:42185_solr-n_0000000004) [n:127.0.0.1:42185_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"pull_replica_error_handling_test_cant_connect_to_pull_replica",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"pull_replica_error_handling_test_cant_connect_to_pull_replica_shard2_replica_p1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42185/solr",
   [junit4]   2>   "type":"PULL"}
   [junit4]   2> 1340098 INFO  (SocketProxy-Acceptor-42185) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=36436,localport=42185], receiveBufferSize:531000
   [junit4]   2> 1340050 INFO  (SocketProxy-Acceptor-44462) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=42272,localport=44462], receiveBufferSize:531000
   [junit4]   2> 1340099 INFO  (SocketProxy-Acceptor-40509) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=46144,localport=40509], receiveBufferSize:531000
   [junit4]   2> 1340099 INFO  (SocketProxy-Acceptor-36469) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44502,localport=36469], receiveBufferSize:531000
   [junit4]   2> 1340113 INFO  (SocketProxy-Acceptor-42185) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=41500,localport=34378], receiveBufferSize=530904
   [junit4]   2> 1340117 INFO  (SocketProxy-Acceptor-40509) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=41510,localport=45172], receiveBufferSize=530904
   [junit4]   2> 1340130 INFO  (SocketProxy-Acceptor-44462) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=34409,localport=55040], receiveBufferSize=530904
   [junit4]   2> 1340164 INFO  (SocketProxy-Acceptor-36469) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=44078,localport=57658], receiveBufferSize=530904
   [junit4]   2> 1340195 INFO  (qtp1278805103-6357) [n:127.0.0.1:42185_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=pull_replica_error_handling_test_cant_connect_to_pull_replica_shard2_replica_p1&action=CREATE&numShards=2&collection=pull_replica_error_handling_test_cant_connect_to_pull_replica&shard=shard2&wt=javabin&version=2&replicaType=PULL
   [junit4]   2> 1340196 INFO  (qtp1278805103-6357) [n:127.0.0.1:42185_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1340216 INFO  (qtp394965859-6445) [n:127.0.0.1:36469_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_p1&action=CREATE&numShards=2&collection=pull_replica_error_handling_test_cant_connect_to_pull_replica&shard=shard1&wt=javabin&version=2&replicaType=PULL
   [junit4]   2> 1340247 INFO  (qtp394965859-6445) [n:127.0.0.1:36469_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1340345 INFO  (qtp1887128045-6389) [n:127.0.0.1:40509_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=pull_replica_error_handling_test_cant_connect_to_pull_replica_shard2_replica_n1&action=CREATE&numShards=2&collection=pull_replica_error_handling_test_cant_connect_to_pull_replica&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1340345 INFO  (qtp1887128045-6389) [n:127.0.0.1:40509_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1340346 INFO  (qtp2025131253-6418) [n:127.0.0.1:44462_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node1&collection.configName=conf&newCollection=true&name=pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_n1&action=CREATE&numShards=2&collection=pull_replica_error_handling_test_cant_connect_to_pull_replica&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1340347 INFO  (qtp2025131253-6418) [n:127.0.0.1:44462_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1340496 INFO  (zkCallback-418-thread-1-processing-n:127.0.0.1:36469_solr) [n:127.0.0.1:36469_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/state.json] for collection [pull_replica_error_handling_test_cant_connect_to_pull_replica] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1340496 INFO  (zkCallback-400-thread-1-processing-n:127.0.0.1:42185_solr) [n:127.0.0.1:42185_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/state.json] for collection [pull_replica_error_handling_test_cant_connect_to_pull_replica] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1340496 INFO  (zkCallback-400-thread-2-processing-n:127.0.0.1:42185_solr) [n:127.0.0.1:42185_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/state.json] for collection [pull_replica_error_handling_test_cant_connect_to_pull_replica] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1340507 INFO  (zkCallback-406-thread-1-processing-n:127.0.0.1:40509_solr) [n:127.0.0.1:40509_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/state.json] for collection [pull_replica_error_handling_test_cant_connect_to_pull_replica] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1340507 INFO  (zkCallback-412-thread-1-processing-n:127.0.0.1:44462_solr) [n:127.0.0.1:44462_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/state.json] for collection [pull_replica_error_handling_test_cant_connect_to_pull_replica] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1340507 INFO  (zkCallback-418-thread-2-processing-n:127.0.0.1:36469_solr) [n:127.0.0.1:36469_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/state.json] for collection [pull_replica_error_handling_test_cant_connect_to_pull_replica] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1340508 INFO  (zkCallback-412-thread-2-processing-n:127.0.0.1:44462_solr) [n:127.0.0.1:44462_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/state.json] for collection [pull_replica_error_handling_test_cant_connect_to_pull_replica] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1340539 INFO  (zkCallback-406-thread-2-processing-n:127.0.0.1:40509_solr) [n:127.0.0.1:40509_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/state.json] for collection [pull_replica_error_handling_test_cant_connect_to_pull_replica] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1341397 INFO  (qtp394965859-6445) [n:127.0.0.1:36469_solr c:pull_replica_error_handling_test_cant_connect_to_pull_replica s:shard1 r:core_node2 x:pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_p1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1341455 INFO  (qtp1278805103-6357) [n:127.0.0.1:42185_solr c:pull_replica_error_handling_test_cant_connect_to_pull_replica s:shard2 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_pull_replica_shard2_replica_p1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1341494 INFO  (qtp2025131253-6418) [n:127.0.0.1:44462_solr c:pull_replica_error_handling_test_cant_connect_to_pull_replica s:shard1 r:core_node1 x:pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1341496 INFO  (qtp394965859-6445) [n:127.0.0.1:36469_solr c:pull_replica_error_handling_test_cant_connect_to_pull_replica s:shard1 r:core_node2 x:pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_p1] o.a.s.s.IndexSchema [pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_p1] Schema name=minimal
   [junit4]   2> 1341522 INFO  (qtp2025131253-6418) [n:127.0.0.1:44462_solr c:pull_replica_error_handling_test_cant_connect_to_pull_replica s:shard1 r:core_node1 x:pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_n1] o.a.s.s.IndexSchema [pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 1341540 INFO  (qtp2025131253-6418) [n:127.0.0.1:44462_solr c:pull_replica_error_handling_test_cant_connect_to_pull_replica s:shard1 r:core_node1 x:pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1341540 INFO  (qtp2025131253-6418) [n:127.0.0.1:44462_solr c:pull_replica_error_handling_test_cant_connect_to_pull_replica s:shard1 r:core_node1 x:pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_n1' using configuration from collection pull_replica_error_handling_test_cant_connect_to_pull_replica, trusted=true
   [junit4]   2> 1341541 INFO  (qtp2025131253-6418) [n:127.0.0.1:44462_solr c:pull_replica_error_handling_test_cant_connect_to_pull_replica s:shard1 r:core_node1 x:pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44462.solr.core.pull_replica_error_handling_test_cant_connect_to_pull_replica.shard1.replica_n1' (registry 'solr.core.pull_replica_error_handling_test_cant_connect_to_pull_replica.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58335475
   [junit4]   2> 1341541 INFO  (qtp2025131253-6418) [n:127.0.0.1:44462_solr c:pull_replica_error_handling_test_cant_connect_to_pull_replica s:shard1 r:core_node1 x:pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1341541 INFO  (qtp2025131253-6418) [n:127.0.0.1:44462_solr c:pull_replica_error_handling_test_cant_connect_to_pull_replica s:shard1 r:core_node1 x:pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_n1] o.a.s.c.SolrCore [[pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_7E60B1F944FBDF16-001/tempDir-001/node2/pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_7E60B1F944FBDF16-001/tempDir-001/node2/./pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_n1/data/]
   [junit4]   2> 1341543 INFO  (qtp1887128045-6389) [n:127.0.0.1:40509_solr c:pull_replica_error_handling_test_cant_connect_to_pull_replica s:shard2 r:core_node3 x:pull_replica_error_handling_test_cant_connect_to_pull_replica_shard2_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1341562 INFO  (qtp394965859-6445) [n:127.0.0.1:36469_solr c:pull_replica_error_handling_test_cant_connect_to_pull_replica s:shard1 r:core_node2 x:pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_p1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1341562 INFO  (qtp394965859-6445) [n:127.0.0.1:36469_solr c:pull_replica_error_handling_test_cant_connect_to_pull_replica s:shard1 r:core_node2 x:pull_replica_error_handling_test_cant_connect_to_p

[...truncated too long message...]

or.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:226)
   [junit4]   2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:211)
   [junit4]   2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:463)
   [junit4]   2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:213)
   [junit4]   2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:594)
   [junit4]   2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:171)
   [junit4]   2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
   [junit4]   2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
   [junit4]   2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
   [junit4]   2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:204)
   [junit4]   2> at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.hdfsFailoverAfterClass(SharedFSAutoReplicaFailoverTest.java:97)
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
   [junit4]   2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
   [junit4]   2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
   [junit4]   2> ... 54 more
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_7E60B1F944FBDF16-001
   [junit4]   2> Jul 14, 2017 12:35:58 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 35 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=Direct), text=FSTOrd50}, docValues:{_version_=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=501, maxMBSortInHeap=6.124080136872814, sim=RandomSimilarity(queryNorm=true): {}, locale=nn-NO, timezone=America/Scoresbysund
   [junit4]   2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=4,threads=2,free=119410144,total=522715136
   [junit4]   2> NOTE: All tests run in this JVM: [DistributedExpandComponentTest, SolrCloudReportersTest, TestScoreJoinQPNoScore, VMParamsZkACLAndCredentialsProvidersTest, TestDFISimilarityFactory, TestLocalFSCloudBackupRestore, TestSmileRequest, V2StandaloneTest, TestCloudRecovery, TestSolrCoreSnapshots, AddSchemaFieldsUpdateProcessorFactoryTest, TestBackupRepositoryFactory, ConjunctionSolrSpellCheckerTest, TestSubQueryTransformerCrossCore, WrapperMergePolicyFactoryTest, TestLMDirichletSimilarityFactory, SpellCheckCollatorTest, TestDocumentBuilder, TestFieldCacheSort, TestFieldTypeCollectionResource, TestUnifiedSolrHighlighter, BooleanFieldTest, TestSolrConfigHandler, TestRandomFaceting, TestSolrQueryParser, CloneFieldUpdateProcessorFactoryTest, TestPushWriter, IndexBasedSpellCheckerTest, ParsingFieldUpdateProcessorsTest, FieldAnalysisRequestHandlerTest, TestBlobHandler, SolrPluginUtilsTest, SolrCLIZkUtilsTest, TestBadConfig, TestExceedMaxTermLength, ChaosMonkeySafeLeaderWithPullReplicasTest, TestWriterPerf, TestNoOpRegenerator, TestCryptoKeys, HdfsChaosMonkeySafeLeaderTest, SolrCoreTest, TestObjectReleaseTracker, ConcurrentDeleteAndCreateCollectionTest, BigEndianAscendingWordDeserializerTest, ConnectionReuseTest, BasicDistributedZkTest, MoveReplicaHDFSUlogDirTest, TestSolrQueryResponse, CloudMLTQParserTest, CdcrRequestHandlerTest, TestReplicaProperties, TestExactSharedStatsCache, TestCloudJSONFacetJoinDomain, HighlighterConfigTest, LeaderFailureAfterFreshStartTest, TestDocTermOrds, ReplicationFactorTest, OverseerStatusTest, TestOrdValues, SolrRequestParserTest, DistributedFacetPivotLongTailTest, TestPHPSerializedResponseWriter, ManagedSchemaRoundRobinCloudTest, RecoveryAfterSoftCommitTest, TemplateUpdateProcessorTest, RAMDirectoryFactoryTest, TestSchemalessBufferedUpdates, TestHighFrequencyDictionaryFactory, TestIBSimilarityFactory, BinaryUpdateRequestHandlerTest, TestMultiValuedNumericRangeQuery, TestSolrFieldCacheBean, TestEmbeddedSolrServerConstructors, BasicAuthIntegrationTest, GraphQueryTest, DebugComponentTest, TestSha256AuthenticationProvider, TestTolerantUpdateProcessorRandomCloud, LeaderElectionIntegrationTest, TestMacros, TestFaceting, TestConfigSetProperties, TestNumericTokenStream, ExitableDirectoryReaderTest, PluginInfoTest, OverseerCollectionConfigSetProcessorTest, DistributedVersionInfoTest, TestDistributedSearch, TestRandomFlRTGCloud, SpatialRPTFieldTypeTest, TestComponentsName, TestBinaryResponseWriter, TestManagedSynonymFilterFactory, XmlUpdateRequestHandlerTest, CdcrBootstrapTest, SortSpecParsingTest, BlobRepositoryCloudTest, TestReplicationHandlerBackup, TestLegacyFieldReuse, MetricsConfigTest, HdfsRecoverLeaseTest, SharedFSAutoReplicaFailoverTest]
   [junit4] Completed [535/727 (3!)] on J0 in 295.05s, 1 test, 1 failure <<< FAILURES!

[...truncated 629 lines...]
   [junit4] JVM J2: stdout was not empty, see: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/temp/junit4-J2-20170713_230347_2671529469476440240751.sysout
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/heapdumps/java_pid6720.hprof ...
   [junit4] Heap dump file created [474916109 bytes in 2.191 secs]
   [junit4] <<< JVM J2: EOF ----

[...truncated 7506 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/build.xml:818: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/build.xml:762: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid6720.hprof

Total time: 448 minutes 30 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-NightlyTests-7.0 - Build # 5 - Still unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.0/5/

3 tests failed.
FAILED:  org.apache.lucene.index.TestStressNRT.test

Error Message:
MockDirectoryWrapper: cannot close: there are still 13 open files: {_ah_LuceneVarGapDocFreqInterval_0.tib=1, _aw_LuceneVarGapDocFreqInterval_0.doc=1, _ah.fdt=1, _av.fdt=1, _ax_LuceneVarGapDocFreqInterval_0.doc=1, _av_LuceneVarGapDocFreqInterval_0.tib=1, _aw.fdt=1, _ax_LuceneVarGapDocFreqInterval_0.tib=1, _ah_LuceneVarGapDocFreqInterval_0.doc=1, _av_LuceneVarGapDocFreqInterval_0.doc=1, _az.cfs=1, _aw_LuceneVarGapDocFreqInterval_0.tib=1, _ax.fdt=1}

Stack Trace:
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still 13 open files: {_ah_LuceneVarGapDocFreqInterval_0.tib=1, _aw_LuceneVarGapDocFreqInterval_0.doc=1, _ah.fdt=1, _av.fdt=1, _ax_LuceneVarGapDocFreqInterval_0.doc=1, _av_LuceneVarGapDocFreqInterval_0.tib=1, _aw.fdt=1, _ax_LuceneVarGapDocFreqInterval_0.tib=1, _ah_LuceneVarGapDocFreqInterval_0.doc=1, _av_LuceneVarGapDocFreqInterval_0.doc=1, _az.cfs=1, _aw_LuceneVarGapDocFreqInterval_0.tib=1, _ax.fdt=1}
        at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
        at org.apache.lucene.index.TestStressNRT.test(TestStressNRT.java:398)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: unclosed IndexInput: _av_LuceneVarGapDocFreqInterval_0.tib
        at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
        at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
        at org.apache.lucene.codecs.blockterms.BlockTermsReader.<init>(BlockTermsReader.java:111)
        at org.apache.lucene.codecs.blockterms.LuceneVarGapDocFreqInterval.fieldsProducer(LuceneVarGapDocFreqInterval.java:115)
        at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:292)
        at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:372)
        at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:112)
        at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:78)
        at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:208)
        at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:281)
        at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:309)
        at org.apache.lucene.index.FrozenBufferedUpdates.apply(FrozenBufferedUpdates.java:297)
        at org.apache.lucene.index.DocumentsWriter$ResolveUpdatesEvent.process(DocumentsWriter.java:723)
        at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5060)
        at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5048)
        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:466)
        at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:293)
        at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:268)
        at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:258)
        at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140)
        at org.apache.lucene.index.TestStressNRT$1.run(TestStressNRT.java:160)


FAILED:  org.apache.solr.core.TestJmxIntegration.testJmxRegistration

Error Message:
org.apache.lucene.store.AlreadyClosedException: this Directory is closed

Stack Trace:
javax.management.RuntimeMBeanException: org.apache.lucene.store.AlreadyClosedException: this Directory is closed
        at __randomizedtesting.SeedInfo.seed([2125E8B3280FE76A:AFF48C89454EBF0F]:0)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
        at org.apache.solr.core.TestJmxIntegration.testJmxRegistration(TestJmxIntegration.java:121)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.lucene.store.AlreadyClosedException: this Directory is closed
        at org.apache.lucene.store.BaseDirectory.ensureOpen(BaseDirectory.java:51)
        at org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:233)
        at org.apache.solr.core.DirectoryFactory.sizeOfDirectory(DirectoryFactory.java:271)
        at org.apache.solr.core.SolrCore.getIndexSize(SolrCore.java:407)
        at org.apache.solr.core.SolrCore.lambda$initializeMetrics$7(SolrCore.java:1137)
        at com.codahale.metrics.JmxReporter$JmxGauge.getValue(JmxReporter.java:188)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
        at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
        at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
        at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
        at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
        at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
        ... 41 more


FAILED:  org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test

Error Message:
Timeout waiting for all live and active

Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
        at __randomizedtesting.SeedInfo.seed([2125E8B3280FE76A:A971D76986F38A92]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:246)
        at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 1772 lines...]
   [junit4] Suite: org.apache.lucene.index.TestStressNRT
   [junit4]   1> READER5: FAILED: unexpected exception
   [junit4]   1> java.lang.AssertionError: No documents or tombstones found for id 115, expected at least 21 reader=StandardDirectoryReader(segments_11:1043:nrt _az(7.0.0):c177/115 _ah(7.0.0):C85/67:delGen=2 _av(7.0.0):C34/26:delGen=1 _aw(7.0.0):C63/47:delGen=1 _ax(7.0.0):C4/3:delGen=1)
   [junit4]   1> at org.junit.Assert.fail(Assert.java:93)
   [junit4]   1> at org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:348)
   [junit4]   2> июл 14, 2017 7:36:22 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[READER5,5,TGRP-TestStressNRT]
   [junit4]   2> java.lang.RuntimeException: java.lang.AssertionError: No documents or tombstones found for id 115, expected at least 21 reader=StandardDirectoryReader(segments_11:1043:nrt _az(7.0.0):c177/115 _ah(7.0.0):C85/67:delGen=2 _av(7.0.0):C34/26:delGen=1 _aw(7.0.0):C63/47:delGen=1 _ax(7.0.0):C4/3:delGen=1)
   [junit4]   2> at __randomizedtesting.SeedInfo.seed([17581654873F0D14]:0)
   [junit4]   2> at org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:377)
   [junit4]   2> Caused by: java.lang.AssertionError: No documents or tombstones found for id 115, expected at least 21 reader=StandardDirectoryReader(segments_11:1043:nrt _az(7.0.0):c177/115 _ah(7.0.0):C85/67:delGen=2 _av(7.0.0):C34/26:delGen=1 _aw(7.0.0):C63/47:delGen=1 _ax(7.0.0):C4/3:delGen=1)
   [junit4]   2> at org.junit.Assert.fail(Assert.java:93)
   [junit4]   2> at org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:348)
   [junit4]   2>
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestStressNRT -Dtests.method=test -Dtests.seed=17581654873F0D14 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/test-data/enwiki.random.lines.txt -Dtests.locale=ru-RU -Dtests.timezone=America/St_Johns -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   2.32s J1 | TestStressNRT.test <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still 13 open files: {_ah_LuceneVarGapDocFreqInterval_0.tib=1, _aw_LuceneVarGapDocFreqInterval_0.doc=1, _ah.fdt=1, _av.fdt=1, _ax_LuceneVarGapDocFreqInterval_0.doc=1, _av_LuceneVarGapDocFreqInterval_0.tib=1, _aw.fdt=1, _ax_LuceneVarGapDocFreqInterval_0.tib=1, _ah_LuceneVarGapDocFreqInterval_0.doc=1, _av_LuceneVarGapDocFreqInterval_0.doc=1, _az.cfs=1, _aw_LuceneVarGapDocFreqInterval_0.tib=1, _ax.fdt=1}
   [junit4]    > at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
   [junit4]    > at org.apache.lucene.index.TestStressNRT.test(TestStressNRT.java:398)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: java.lang.RuntimeException: unclosed IndexInput: _av_LuceneVarGapDocFreqInterval_0.tib
   [junit4]    > at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
   [junit4]    > at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
   [junit4]    > at org.apache.lucene.codecs.blockterms.BlockTermsReader.<init>(BlockTermsReader.java:111)
   [junit4]    > at org.apache.lucene.codecs.blockterms.LuceneVarGapDocFreqInterval.fieldsProducer(LuceneVarGapDocFreqInterval.java:115)
   [junit4]    > at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:292)
   [junit4]    > at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:372)
   [junit4]    > at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:112)
   [junit4]    > at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:78)
   [junit4]    > at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:208)
   [junit4]    > at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:281)
   [junit4]    > at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:309)
   [junit4]    > at org.apache.lucene.index.FrozenBufferedUpdates.apply(FrozenBufferedUpdates.java:297)
   [junit4]    > at org.apache.lucene.index.DocumentsWriter$ResolveUpdatesEvent.process(DocumentsWriter.java:723)
   [junit4]    > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5060)
   [junit4]    > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5048)
   [junit4]    > at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:466)
   [junit4]    > at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:293)
   [junit4]    > at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:268)
   [junit4]    > at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:258)
   [junit4]    > at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140)
   [junit4]    > at org.apache.lucene.index.TestStressNRT$1.run(TestStressNRT.java:160)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=14768, name=READER5, state=RUNNABLE, group=TGRP-TestStressNRT]
   [junit4]    > Caused by: java.lang.RuntimeException: java.lang.AssertionError: No documents or tombstones found for id 115, expected at least 21 reader=StandardDirectoryReader(segments_11:1043:nrt _az(7.0.0):c177/115 _ah(7.0.0):C85/67:delGen=2 _av(7.0.0):C34/26:delGen=1 _aw(7.0.0):C63/47:delGen=1 _ax(7.0.0):C4/3:delGen=1)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([17581654873F0D14]:0)
   [junit4]    > at org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:377)
   [junit4]    > Caused by: java.lang.AssertionError: No documents or tombstones found for id 115, expected at least 21 reader=StandardDirectoryReader(segments_11:1043:nrt _az(7.0.0):c177/115 _ah(7.0.0):C85/67:delGen=2 _av(7.0.0):C34/26:delGen=1 _aw(7.0.0):C63/47:delGen=1 _ax(7.0.0):C4/3:delGen=1)
   [junit4]    > at org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:348)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{}, maxPointsInLeafNode=733, maxMBSortInHeap=6.40435406004078, sim=RandomSimilarity(queryNorm=true): {id=DFR I(F)L3(800.0)}, locale=ru-RU, timezone=America/St_Johns
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=4,threads=1,free=219031856,total=534249472
   [junit4]   2> NOTE: All tests run in this JVM: [TestDirectory, TestDocValuesQueries, TestPointQueries, TestBM25Similarity, TestMmapDirectory, TestPhraseQuery, TestFSTs, TestSpanOrQuery, TestIntroSorter, TestCrash, TestTermsEnum2, TestPhrasePrefixQuery, TestConcurrentMergeScheduler, TestLucene70DocValuesFormat, TestPerFieldDocValuesFormat, TestMinimize, TestTransactionRollback, TestRAMDirectory, TestLucene60FieldInfoFormat, TestRegexpRandom2, TestSegmentInfos, TestTermdocPerf, TestPositionIncrement, TestIndexWriterDelete, TestIndexWriterOnJRECrash, TestFilterDirectoryReader, TestFixedBitDocIdSet, TestReusableStringReader, TestSwappedIndexFiles, TestBytesRefArray, TestSumDocFreq, TestTieredMergePolicy, TestRecyclingIntBlockAllocator, TestByteSlices, TestSpanExplanations, TestSpanContainQuery, Test2BPostings, TestCharArraySet, TestSingleInstanceLockFactory, TestCheckIndex, TestDoubleValuesSource, TestLockFactory, TestElevationComparator, TestRollback, Test2BSortedDocValuesFixedSorted, TestMatchNoDocsQuery, TestBoostQuery, TestAddIndexes, TestTermVectorsReader, LimitedFiniteStringsIteratorTest, TestPointValues, TestIndexSorting, TestTimeLimitingCollector, TestSearcherManager, TestMultiPhraseEnum, TestTransactions, TestDemo, TestCustomSearcherSort, TestAllFilesDetectTruncation, TestMultiTermConstantScore, TestWildcard, TestDocInverterPerFieldErrorInfo, TestFieldType, TestFieldInvertState, TestAutomatonQueryUnicode, TestPriorityQueue, TestBufferedChecksum, TestMSBRadixSorter, TestMixedDocValuesUpdates, TestSameScoresWithThreads, Test2BDocs, TestLevenshteinAutomata, TestMergeSchedulerExternal, TestSimpleExplanationsWithFillerDocs, TestStressNRT]
   [junit4] Completed [454/454 (1!)] on J1 in 2.33s, 1 test, 1 error <<< FAILURES!

[...truncated 9797 lines...]
   [junit4] Suite: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_2125E8B3280FE76A-001/init-core-data-001
   [junit4]   2> 252103 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 252103 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 252105 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
   [junit4]   2> 252105 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 252155 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 252163 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 252165 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 252181 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_56173_hdfs____.iloqmi/webapp
   [junit4]   2> 252551 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:56173
   [junit4]   2> 252623 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 252624 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 252639 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_46922_datanode____idpala/webapp
   [junit4]   2> 253070 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46922
   [junit4]   2> 253122 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 253124 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 253137 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_43596_datanode____83uzdm/webapp
   [junit4]   2> 253269 INFO  (IPC Server handler 2 on 37435) [    ] BlockStateChange BLOCK* processReport: from storage DS-e906f4ea-487e-4307-bf07-0b05e4c8e6fe node DatanodeRegistration(127.0.0.1:52881, datanodeUuid=a7353420-502d-40a9-8290-8a5f5fcb0b1a, infoPort=56938, infoSecurePort=0, ipcPort=56868, storageInfo=lv=-56;cid=testClusterID;nsid=1692542618;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 253269 INFO  (IPC Server handler 2 on 37435) [    ] BlockStateChange BLOCK* processReport: from storage DS-fe990204-45f6-498e-bc83-cd995920c9ce node DatanodeRegistration(127.0.0.1:52881, datanodeUuid=a7353420-502d-40a9-8290-8a5f5fcb0b1a, infoPort=56938, infoSecurePort=0, ipcPort=56868, storageInfo=lv=-56;cid=testClusterID;nsid=1692542618;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 253558 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[2125E8B3280FE76A]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43596
   [junit4]   2> 253676 INFO  (IPC Server handler 3 on 37435) [    ] BlockStateChange BLOCK* processReport: from storage DS-d15e90f7-3214-48ca-b687-00114f319957 node DatanodeRegistration(127.0.0.1:38667, datanodeUuid=7d70cca6-2bda-4d7b-a680-c63d877139b2, infoPort=58664, infoSecurePort=0, ipcPort=57094, storageInfo=lv=-56;cid=testClusterID;nsid=1692542618;c=0), blocks: 0, hasStaleStorage: true, processing time: 1 msecs
   [junit4]   2> 253676 INFO  (IPC Server handler 3 on 37435) [    ] BlockStateChange BLOCK* processReport: from storage DS-67c6100e-131e-4554-960f-39f38fb1479b node DatanodeRegistration(127.0.0.1:38667, datanodeUuid=7d70cca6-2bda-4d7b-a680-c63d877139b2, infoPort=58664, infoSecurePort=0, ipcPort=57094, storageInfo=lv=-56;cid=testClusterID;nsid=1692542618;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 253735 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 253735 INFO  (Thread-1225) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 253735 INFO  (Thread-1225) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 253738 ERROR (Thread-1225) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 253835 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.ZkTestServer start zk server on port:57346
   [junit4]   2> 253843 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15d4394125a0000, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 253857 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 253860 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 253862 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 253863 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 253871 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 253873 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 253875 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 253877 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 253879 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 253880 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 253882 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 253885 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 254022 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 254024 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@70d7d04f{/,null,AVAILABLE}
   [junit4]   2> 254025 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2cbeecee{HTTP/1.1,[http/1.1]}{127.0.0.1:51288}
   [junit4]   2> 254025 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.e.j.s.Server Started @257245ms
   [junit4]   2> 254025 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=51288, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_2125E8B3280FE76A-001/control-001/cores}
   [junit4]   2> 254026 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 254026 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 254026 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 254026 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 254026 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-15T00:09:24.091Z
   [junit4]   2> 254035 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 254035 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_2125E8B3280FE76A-001/control-001/solr.xml
   [junit4]   2> 254044 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@12b64677, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 254049 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57346/solr
   [junit4]   2> 254204 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:51288_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 254205 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:51288_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51288_
   [junit4]   2> 254206 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:51288_    ] o.a.s.c.Overseer Overseer (id=98309070114127876-127.0.0.1:51288_-n_0000000000) starting
   [junit4]   2> 254215 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:51288_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51288_
   [junit4]   2> 254217 INFO  (zkCallback-189-thread-1-processing-n:127.0.0.1:51288_) [n:127.0.0.1:51288_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 254293 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:51288_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12b64677
   [junit4]   2> 254298 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:51288_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12b64677
   [junit4]   2> 254298 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:51288_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12b64677
   [junit4]   2> 254300 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:51288_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_2125E8B3280FE76A-001/control-001/cores
   [junit4]   2> 254321 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 254322 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:57346/solr ready
   [junit4]   2> 254326 INFO  (qtp1016457547-2302) [n:127.0.0.1:51288_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:51288_&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 254329 INFO  (OverseerThreadFactory-470-thread-1-processing-n:127.0.0.1:51288_) [n:127.0.0.1:51288_    ] o.a.s.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 254437 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 254437 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 254543 INFO  (zkCallback-189-thread-1-processing-n:127.0.0.1:51288_) [n:127.0.0.1:51288_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 255456 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 255470 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 255555 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 255575 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 255575 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12b64677
   [junit4]   2> 255575 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:37435/solr_hdfs_home
   [junit4]   2> 255576 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 255576 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 255576 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_2125E8B3280FE76A-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[hdfs://localhost:37435/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 255578 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37435/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
   [junit4]   2> 255586 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 255586 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
   [junit4]   2> 255593 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 255594 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37435/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 255613 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37435/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 255620 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 255620 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
   [junit4]   2> 255627 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 255628 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=35, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.8505859375, noCFSRatio=1.0]
   [junit4]   2> 255648 INFO  (IPC Server handler 5 on 37435) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:52881 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-d15e90f7-3214-48ca-b687-00114f319957:NORMAL:127.0.0.1:38667|RBW], ReplicaUC[[DISK]DS-fe990204-45f6-498e-bc83-cd995920c9ce:NORMAL:127.0.0.1:52881|FINALIZED]]} size 0
   [junit4]   2> 255649 INFO  (IPC Server handler 6 on 37435) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:38667 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-fe990204-45f6-498e-bc83-cd995920c9ce:NORMAL:127.0.0.1:52881|FINALIZED], ReplicaUC[[DISK]DS-67c6100e-131e-4554-960f-39f38fb1479b:NORMAL:127.0.0.1:38667|FINALIZED]]} size 0
   [junit4]   2> 255656 WARN  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 255714 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 255714 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 255714 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 255726 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 255726 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 255729 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@219d810e
   [junit4]   2> 255738 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@358ee747[control_collection_shard1_replica_n1] main]
   [junit4]   2> 255739 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 255740 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 255741 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 255743 INFO  (searcherExecutor-473-thread-1-processing-n:127.0.0.1:51288_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@358ee747[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 255743 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1572945123929489408
   [junit4]   2> 255750 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 255750 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 255750 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51288/control_collection_shard1_replica_n1/
   [junit4]   2> 255750 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 255750 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:51288/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 255750 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 255755 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51288/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 255858 INFO  (zkCallback-189-thread-1-processing-n:127.0.0.1:51288_) [n:127.0.0.1:51288_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 255906 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 255909 INFO  (qtp1016457547-2304) [n:127.0.0.1:51288_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1472
   [junit4]   2> 255913 INFO  (qtp1016457547-2302) [n:127.0.0.1:51288_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 256011 INFO  (zkCallback-189-thread-1-processing-n:127.0.0.1:51288_) [n:127.0.0.1:51288_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 256331 INFO  (OverseerCollectionConfigSetProcessor-98309070114127876-127.0.0.1:51288_-n_0000000000) [n:127.0.0.1:51288_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 256913 INFO  (qtp1016457547-2302) [n:127.0.0.1:51288_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:51288_&wt=javabin&version=2} status=0 QTime=2587
   [junit4]   2> 256921 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 256922 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:57346/solr ready
   [junit4]   2> 256922 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 256924 INFO  (qtp1016457547-2303) [n:127.0.0.1:51288_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 256926 INFO  (OverseerThreadFactory-470-thread-2-processing-n:127.0.0.1:51288_) [n:127.0.0.1:51288_    ] o.a.s.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 256927 WARN  (OverseerThreadFactory-470-thread-2-processing-n:127.0.0.1:51288_) [n:127.0.0.1:51288_    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 257140 INFO  (qtp1016457547-2303) [n:127.0.0.1:51288_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 257140 INFO  (qtp1016457547-2303) [n:127.0.0.1:51288_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=216
   [junit4]   2> 257280 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_2125E8B3280FE76A-001/shard-1-001 of type NRT
   [junit4]   2> 257282 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 257284 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@228af05a{/,null,AVAILABLE}
   [junit4]   2> 257284 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@549c2c49{HTTP/1.1,[http/1.1]}{127.0.0.1:36079}
   [junit4]   2> 257284 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.e.j.s.Server Started @260504ms
   [junit4]   2> 257285 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/, hostPort=36079, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_2125E8B3280FE76A-001/shard-1-001/cores}
   [junit4]   2> 257285 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 257286 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 257286 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 257286 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 257286 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-15T00:09:27.351Z
   [junit4]   2> 257291 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 257291 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_2125E8B3280FE76A-001/shard-1-001/solr.xml
   [junit4]   2> 257297 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@12b64677, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 257300 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57346/solr
   [junit4]   2> 257323 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:36079_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 257327 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:36079_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 257330 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:36079_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36079_
   [junit4]   2> 257331 INFO  (zkCallback-196-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 257332 INFO  (zkCallback-189-thread-1-processing-n:127.0.0.1:51288_) [n:127.0.0.1:51288_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 257332 INFO  (zkCallback-201-thread-1-processing-n:127.0.0.1:36079_) [n:127.0.0.1:36079_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 257427 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:36079_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12b64677
   [junit4]   2> 257433 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:36079_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12b64677
   [junit4]   2> 257433 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:36079_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12b64677
   [junit4]   2> 257434 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:36079_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_2125E8B3280FE76A-001/shard-1-001/cores
   [junit4]   2> 257462 INFO  (qtp689432806-2353) [n:127.0.0.1:36079_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:36079_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 257465 INFO  (OverseerCollectionConfigSetProcessor-98309070114127876-127.0.0.1:51288_-n_0000000000) [n:127.0.0.1:51288_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 257465 INFO  (OverseerThreadFactory-470-thread-3-processing-n:127.0.0.1:51288_) [n:127.0.0.1:51288_    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:36079_ for creating new replica
   [junit4]   2> 257468 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n0&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 257468 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 258486 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 258500 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.s.IndexSchema [collection1_shard2_replica_n0] Schema name=test
   [junit4]   2> 258588 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 258608 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n0' using configuration from collection collection1, trusted=true
   [junit4]   2> 258608 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n0' (registry 'solr.core.collection1.shard2.replica_n0') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12b64677
   [junit4]   2> 258608 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:37435/solr_hdfs_home
   [junit4]   2> 258609 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 258609 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 258609 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.SolrCore [[collection1_shard2_replica_n0] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_2125E8B3280FE76A-001/shard-1-001/cores/collection1_shard2_replica_n0], dataDir=[hdfs://localhost:37435/solr_hdfs_home/collection1/core_node1/data/]
   [junit4]   2> 258610 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37435/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
   [junit4]   2> 258619 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 258619 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
   [junit4]   2> 258623 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 258624 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37435/solr_hdfs_home/collection1/core_node1/data
   [junit4]   2> 258642 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37435/solr_hdfs_home/collection1/core_node1/data/index
   [junit4]   2> 258649 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 258649 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
   [junit4]   2> 258653 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 258653 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=35, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.8505859375, noCFSRatio=1.0]
   [junit4]   2> 258680 INFO  (IPC Server handler 8 on 37435) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:52881 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-d15e90f7-3214-48ca-b687-00114f319957:NORMAL:127.0.0.1:38667|RBW], ReplicaUC[[DISK]DS-e906f4ea-487e-4307-bf07-0b05e4c8e6fe:NORMAL:127.0.0.1:52881|FINALIZED]]} size 0
   [junit4]   2> 258681 INFO  (IPC Server handler 7 on 37435) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:38667 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-d15e90f7-3214-48ca-b687-00114f319957:NORMAL:127.0.0.1:38667|RBW], ReplicaUC[[DISK]DS-e906f4ea-487e-4307-bf07-0b05e4c8e6fe:NORMAL:127.0.0.1:52881|FINALIZED]]} size 0
   [junit4]   2> 258687 WARN  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 258755 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 258755 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 258755 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 258767 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 258767 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 258770 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@64038de7
   [junit4]   2> 258777 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.s.SolrIndexSearcher Opening [Searcher@15abf7bb[collection1_shard2_replica_n0] main]
   [junit4]   2> 258778 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 258778 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 258779 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 258783 INFO  (searcherExecutor-484-thread-1-processing-n:127.0.0.1:36079_ x:collection1_shard2_replica_n0 s:shard2 c:collection1) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.SolrCore [collection1_shard2_replica_n0] Registered new searcher Searcher@15abf7bb[collection1_shard2_replica_n0] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 258783 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1572945127117160448
   [junit4]   2> 258790 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 258790 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 258790 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36079/collection1_shard2_replica_n0/
   [junit4]   2> 258790 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 258790 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.SyncStrategy http://127.0.0.1:36079/collection1_shard2_replica_n0/ has no replicas
   [junit4]   2> 258790 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 258794 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36079/collection1_shard2_replica_n0/ shard2
   [junit4]   2> 258946 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 258952 INFO  (qtp689432806-2351) [n:127.0.0.1:36079_ c:collection1 s:shard2  x:collection1_shard2_replica_n0] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n0&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1484
   [junit4]   2> 258955 INFO  (qtp689432806-2353) [n:127.0.0.1:36079_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:36079_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1493
   [junit4]   2> 259095 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_2125E8B3280FE76A-001/shard-2-001 of type NRT
   [junit4]   2> 259095 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 259097 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5b0341cf{/,null,AVAILABLE}
   [junit4]   2> 259097 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@581c6343{HTTP/1.1,[http/1.1]}{127.0.0.1:55878}
   [junit4]   2> 259098 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.e.j.s.Server Started @262318ms
   [junit4]   2> 259098 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=55878, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_2125E8B3280FE76A-001/shard-2-001/cores}
   [junit4]   2> 259098 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 259099 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 259099 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 259099 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 259099 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-07-15T00:09:29.164Z
   [junit4]   2> 259104 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 259104 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_2125E8B3280FE76A-001/shard-2-001/solr.xml
   [junit4]   2> 259110 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@12b64677, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 259114 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57346/solr
   [junit4]   2> 259129 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:55878_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 259132 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:55878_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 259136 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:55878_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55878_
   [junit4]   2> 259137 INFO  (zkCallback-201-thread-1-processing-n:127.0.0.1:36079_) [n:127.0.0.1:36079_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 259138 INFO  (zkCallback-196-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 259138 INFO  (zkCallback-189-thread-2-processing-n:127.0.0.1:51288_) [n:127.0.0.1:51288_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 259139 INFO  (zkCallback-207-thread-1-processing-n:127.0.0.1:55878_) [n:127.0.0.1:55878_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 259249 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:55878_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12b64677
   [junit4]   2> 259255 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:55878_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12b64677
   [junit4]   2> 259255 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:55878_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12b64677
   [junit4]   2> 259257 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[2125E8B3280FE76A]) [n:127.0.0.1:55878_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_2125E8B3280FE76A-001/shard-2-001/cores
   [junit4]   2> 259289 INFO  (qtp689432806-2354) [n:127.0.0.1:36079_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:55878_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 259292 INFO  (OverseerThreadFactory-470-thread-4-processing-n:127.0.0.1:51288_) [n:127.0.0.1:51288_    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:55878_ for creating new replica
   [junit4]   2> 259292 INFO  (OverseerCollectionConfigSetProcessor-98309070114127876-127.0.0.1:51288_-n_0000000000) [n:127.0.0.1:51288_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 259297 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n0&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 259298 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 260316 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 260330 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.s.IndexSchema [collection1_shard1_replica_n0] Schema name=test
   [junit4]   2> 260417 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 260438 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n0' using configuration from collection collection1, trusted=true
   [junit4]   2> 260438 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n0' (registry 'solr.core.collection1.shard1.replica_n0') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12b64677
   [junit4]   2> 260438 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:37435/solr_hdfs_home
   [junit4]   2> 260438 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 260439 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 260439 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.SolrCore [[collection1_shard1_replica_n0] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_2125E8B3280FE76A-001/shard-2-001/cores/collection1_shard1_replica_n0], dataDir=[hdfs://localhost:37435/solr_hdfs_home/collection1/core_node2/data/]
   [junit4]   2> 260440 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37435/solr_hdfs_home/collection1/core_node2/data/snapshot_metadata
   [junit4]   2> 260449 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 260449 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
   [junit4]   2> 260453 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 260454 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37435/solr_hdfs_home/collection1/core_node2/data
   [junit4]   2> 260474 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37435/solr_hdfs_home/collection1/core_node2/data/index
   [junit4]   2> 260481 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 260481 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
   [junit4]   2> 260484 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 260485 INFO  (qtp1418706108-2390) [n:127.0.0.1:55878_ c:collection1 s:shard1  x:collection1_shard1_replica_n0] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=35, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.8505859375, noCFSRatio=1.0]
   [junit4]   2> 260502 INFO  (IPC Server handler 9 on 37435) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:52881 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-67c6100e-131e-4554-960f-39f38fb1479b:NORMAL:127.0.0.1:38667|RBW], ReplicaUC[[DISK]DS-fe990204-45f6-498e-bc83-cd995920c9ce:NORMAL:127.0.0.1:52881|FINALIZED]]} size 0
   [junit4]   2> 260503 INFO  (IPC Server handler 0 on 37435) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:38667 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNo

[...truncated too long message...]

-TestJmxIntegration.testJmxUpdate-seed#[2125E8B3280FE76A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 3211590 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[2125E8B3280FE76A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@5c146323 commitCommandVersion:0
   [junit4]   2> 3211604 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[2125E8B3280FE76A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@98cda18[collection1] main]
   [junit4]   2> 3211604 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[2125E8B3280FE76A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 3211608 INFO  (searcherExecutor-8526-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@98cda18[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):c1)))}
   [junit4]   2> 3211612 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[2125E8B3280FE76A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 21
   [junit4]   2> 3211612 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[2125E8B3280FE76A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testJmxUpdate
   [junit4]   2> 3211616 INFO  (TEST-TestJmxIntegration.testJmxRegistration-seed#[2125E8B3280FE76A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testJmxRegistration
   [junit4]   2> 3211676 INFO  (TEST-TestJmxIntegration.testJmxRegistration-seed#[2125E8B3280FE76A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testJmxRegistration
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestJmxIntegration -Dtests.method=testJmxRegistration -Dtests.seed=2125E8B3280FE76A -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/test-data/enwiki.random.lines.txt -Dtests.locale=ar-SA -Dtests.timezone=America/Whitehorse -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.07s J2 | TestJmxIntegration.testJmxRegistration <<<
   [junit4]    > Throwable #1: javax.management.RuntimeMBeanException: org.apache.lucene.store.AlreadyClosedException: this Directory is closed
   [junit4]    > at __randomizedtesting.SeedInfo.seed([2125E8B3280FE76A:AFF48C89454EBF0F]:0)
   [junit4]    > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
   [junit4]    > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
   [junit4]    > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
   [junit4]    > at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
   [junit4]    > at org.apache.solr.core.TestJmxIntegration.testJmxRegistration(TestJmxIntegration.java:121)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: org.apache.lucene.store.AlreadyClosedException: this Directory is closed
   [junit4]    > at org.apache.lucene.store.BaseDirectory.ensureOpen(BaseDirectory.java:51)
   [junit4]    > at org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:233)
   [junit4]    > at org.apache.solr.core.DirectoryFactory.sizeOfDirectory(DirectoryFactory.java:271)
   [junit4]    > at org.apache.solr.core.SolrCore.getIndexSize(SolrCore.java:407)
   [junit4]    > at org.apache.solr.core.SolrCore.lambda$initializeMetrics$7(SolrCore.java:1137)
   [junit4]    > at com.codahale.metrics.JmxReporter$JmxGauge.getValue(JmxReporter.java:188)
   [junit4]    > at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
   [junit4]    > at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
   [junit4]    > at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
   [junit4]    > at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
   [junit4]    > at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
   [junit4]    > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
   [junit4]    > ... 41 more
   [junit4]   2> 3211680 INFO  (SUITE-TestJmxIntegration-seed#[2125E8B3280FE76A]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 3211680 INFO  (SUITE-TestJmxIntegration-seed#[2125E8B3280FE76A]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=892545155
   [junit4]   2> 3211680 INFO  (SUITE-TestJmxIntegration-seed#[2125E8B3280FE76A]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 3211691 INFO  (SUITE-TestJmxIntegration-seed#[2125E8B3280FE76A]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 3211694 INFO  (SUITE-TestJmxIntegration-seed#[2125E8B3280FE76A]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 3211699 INFO  (SUITE-TestJmxIntegration-seed#[2125E8B3280FE76A]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3211700 INFO  (coreCloseExecutor-8530-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@58e3282c
   [junit4]   2> 3211702 INFO  (coreCloseExecutor-8530-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1491281964
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.0/checkout/solr/build/solr-core/test/J2/temp/solr.core.TestJmxIntegration_2125E8B3280FE76A-001
   [junit4]   2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=false): {}, locale=ar-SA, timezone=America/Whitehorse
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=4,threads=1,free=252299992,total=532152320
   [junit4]   2> NOTE: All tests run in this JVM: [TestNumericTerms32, ResponseLogComponentTest, DistributedQueueTest, RollingRestartTest, QueryResultKeyTest, PreAnalyzedFieldTest, HardAutoCommitTest, StressHdfsTest, DeleteInactiveReplicaTest, TestOnReconnectListenerSupport, DirectUpdateHandlerTest, SegmentsInfoRequestHandlerTest, TestTolerantUpdateProcessorCloud, MetricUtilsTest, SimpleCollectionCreateDeleteTest, PingRequestHandlerTest, PrimitiveFieldTypeTest, AliasIntegrationTest, ShardSplitTest, SmileWriterTest, ShowFileRequestHandlerTest, DateMathParserTest, HdfsNNFailoverTest, XsltUpdateRequestHandlerTest, TestRecovery, SimplePostToolTest, TestSolrCloudWithKerberosAlt, TestPullReplicaErrorHandling, DataDrivenBlockJoinTest, TestPerFieldSimilarityWithDefaultOverride, RegexBoostProcessorTest, SolrInfoBeanTest, TriLevelCompositeIdRoutingTest, DeleteReplicaTest, UniqFieldsUpdateProcessorFactoryTest, URLClassifyProcessorTest, OverseerRolesTest, MetricsHandlerTest, BadCopyFieldTest, HdfsLockFactoryTest, TestLegacyNumericRangeQueryBuilder, TestSimpleQParserPlugin, HdfsRestartWhileUpdatingTest, TestCollationFieldDocValues, TestCollapseQParserPlugin, ClassificationUpdateProcessorTest, TestPolicyCloud, TestDistribDocBasedVersion, TestSuggestSpellingConverter, TestSolrCoreProperties, TestQueryUtils, CoreAdminRequestStatusTest, AddBlockUpdateTest, SuggesterFSTTest, TestPivotHelperCode, TestManagedStopFilterFactory, SearchHandlerTest, TimeZoneUtilsTest, TestConfigSetImmutable, BJQParserTest, ConnectionManagerTest, NotRequiredUniqueKeyTest, SearchHandlerTest, AtomicUpdatesTest, OverseerTest, TestSolrCLIRunExample, BadIndexSchemaTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestAnalyzeInfixSuggestions, DocumentBuilderTest, DistributedFacetPivotLargeTest, BasicDistributedZk2Test, ResponseHeaderTest, TestLFUCache, CdcrReplicationHandlerTest, LeaderFailoverAfterPartitionTest, TestSchemaManager, TestSolrDeletionPolicy2, SolrXmlInZkTest, SimpleFacetsTest, SpatialHeatmapFacetsTest, RequiredFieldsTest, TestLegacyField, TestStressUserVersions, HdfsTlogReplayBufferedWhileIndexingTest, TestMiniSolrCloudCluster, LeaderInitiatedRecoveryOnShardRestartTest, JvmMetricsTest, TestStressReorder, TestDistributedMissingSort, TestQuerySenderListener, NumericFieldsTest, TestCustomSort, TestDistributedStatsComponentCardinality, CollectionStateFormat2Test, TestSchemaVersionResource, TestRequestStatusCollectionAPI, TestRandomCollapseQParserPlugin, ChaosMonkeyNothingIsSafeWithPullReplicasTest, TestPayloadCheckQParserPlugin, TestNestedDocsSort, TestFieldCacheReopen, TestRebalanceLeaders, TestConfigSetsAPIZkFailure, TestMultiWordSynonyms, CopyFieldTest, TestSurroundQueryParser, PKIAuthenticationIntegrationTest, ZkStateWriterTest, SolrSlf4jReporterTest, TestLeaderInitiatedRecoveryThread, InfixSuggestersTest, TestRuleBasedAuthorizationPlugin, BitVectorTest, SolrTestCaseJ4Test, CachingDirectoryFactoryTest, SpellCheckComponentTest, TestGroupingSearch, TestSort, TestFunctionQuery, TestLazyCores, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, SpellCheckCollatorTest, TestTrie, PolyFieldTest, SchemaVersionSpecificBehaviorTest, TestUpdate, DirectUpdateHandlerOptimizeTest, SortByFunctionTest, DocValuesMultiTest, DebugComponentTest, DisMaxRequestHandlerTest, DirectSolrSpellCheckerTest, TermsComponentTest, TestIndexingPerformance, FastVectorHighlighterTest, IndexSchemaRuntimeFieldTest, TestJmxIntegration]
   [junit4] Completed [424/727 (2!)] on J2 in 0.80s, 3 tests, 1 error, 1 skipped <<< FAILURES!

[...truncated 44322 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Loading...