[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-11-ea+28) - Build # 2735 - Unstable!

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-11-ea+28) - Build # 2735 - Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/2735/
Java: 64bit/jdk-11-ea+28 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

21 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.handler.extraction.ExtractingRequestHandlerTest

Error Message:
SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.

Stack Trace:
java.lang.AssertionError: SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.
        at __randomizedtesting.SeedInfo.seed([CB2703E0A2284453]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:44)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.extraction.ExtractingRequestHandlerTest

Error Message:
SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.

Stack Trace:
java.lang.AssertionError: SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.
        at __randomizedtesting.SeedInfo.seed([CB2703E0A2284453]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:44)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.extraction.ExtractingRequestHandlerTest

Error Message:
SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.

Stack Trace:
java.lang.AssertionError: SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.
        at __randomizedtesting.SeedInfo.seed([CB2703E0A2284453]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:44)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.extraction.ExtractingRequestHandlerTest

Error Message:
SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.

Stack Trace:
java.lang.AssertionError: SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.
        at __randomizedtesting.SeedInfo.seed([CB2703E0A2284453]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:44)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.extraction.ExtractingRequestHandlerTest

Error Message:
SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.

Stack Trace:
java.lang.AssertionError: SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.
        at __randomizedtesting.SeedInfo.seed([CB2703E0A2284453]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:44)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.client.solrj.io.stream.MathExpressionTest.testConvertEvaluator

Error Message:
--> https://127.0.0.1:43623/solr/collection1:java.util.concurrent.ExecutionException: java.io.IOException: --> https://127.0.0.1:43631/solr/collection1_shard1_replica_n1/:java.util.concurrent.ExecutionException: java.io.IOException: params q=*:*&partitionKeys=miles_i&sort=miles_i+asc&fl=miles_i&distrib=false

Stack Trace:
java.io.IOException: --> https://127.0.0.1:43623/solr/collection1:java.util.concurrent.ExecutionException: java.io.IOException: --> https://127.0.0.1:43631/solr/collection1_shard1_replica_n1/:java.util.concurrent.ExecutionException: java.io.IOException: params q=*:*&partitionKeys=miles_i&sort=miles_i+asc&fl=miles_i&distrib=false
        at __randomizedtesting.SeedInfo.seed([52F00B199ABF9487:31026EC2AA4A69F8]:0)
        at org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:215)
        at org.apache.solr.client.solrj.io.stream.MathExpressionTest.getTuples(MathExpressionTest.java:4736)
        at org.apache.solr.client.solrj.io.stream.MathExpressionTest.testConvertEvaluator(MathExpressionTest.java:4711)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.TestSkipOverseerOperations.testSkipDownOperations

Error Message:
Expected 2x2 for collection: collection2 null Live Nodes: [127.0.0.1:34009_solr, 127.0.0.1:37117_solr, 127.0.0.1:40135_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Expected 2x2 for collection: collection2
null
Live Nodes: [127.0.0.1:34009_solr, 127.0.0.1:37117_solr, 127.0.0.1:40135_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([3E9DE8CFC015A7BB:F90F9A893D591C6D]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.TestSkipOverseerOperations.testSkipDownOperations(TestSkipOverseerOperations.java:104)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeAddedTriggerWithAddReplicaPreferredOp_2Shard

Error Message:
Timed out waiting for replicas of new collection to be active null Live Nodes: [127.0.0.1:37567_solr, 127.0.0.1:44497_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for replicas of new collection to be active
null
Live Nodes: [127.0.0.1:37567_solr, 127.0.0.1:44497_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([3E9DE8CFC015A7BB:8706D3105349747D]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.nodeAddedTriggerWithAddReplicaPreferredOp(ComputePlanActionTest.java:585)
        at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeAddedTriggerWithAddReplicaPreferredOp_2Shard(ComputePlanActionTest.java:524)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeLostTriggerWithDeleteNodePreferredOp

Error Message:
Timed out waiting for replicas of new collection to be active null Live Nodes: [127.0.0.1:40229_solr, 127.0.0.1:44693_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for replicas of new collection to be active
null
Live Nodes: [127.0.0.1:40229_solr, 127.0.0.1:44693_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([3E9DE8CFC015A7BB:A1324C2CB7AD3ED2]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeLostTriggerWithDeleteNodePreferredOp(ComputePlanActionTest.java:655)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeAddedTriggerWithAddReplicaPreferredOp_2Shard

Error Message:
Timed out waiting for replicas of new collection to be active null Live Nodes: [127.0.0.1:36055_solr, 127.0.0.1:40597_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for replicas of new collection to be active
null
Live Nodes: [127.0.0.1:36055_solr, 127.0.0.1:40597_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([3E9DE8CFC015A7BB:8706D3105349747D]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.nodeAddedTriggerWithAddReplicaPreferredOp(ComputePlanActionTest.java:585)
        at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeAddedTriggerWithAddReplicaPreferredOp_2Shard(ComputePlanActionTest.java:524)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeLostTriggerWithDeleteNodePreferredOp

Error Message:
Timed out waiting for replicas of new collection to be active null Live Nodes: [127.0.0.1:35255_solr, 127.0.0.1:44937_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for replicas of new collection to be active
null
Live Nodes: [127.0.0.1:35255_solr, 127.0.0.1:44937_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([3E9DE8CFC015A7BB:A1324C2CB7AD3ED2]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeLostTriggerWithDeleteNodePreferredOp(ComputePlanActionTest.java:655)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeAdded

Error Message:
Timed out waiting for replicas of new collection to be active null Live Nodes: [127.0.0.1:44937_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for replicas of new collection to be active
null
Live Nodes: [127.0.0.1:44937_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([3E9DE8CFC015A7BB:5B5EBEB862B60FB8]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeAdded(ComputePlanActionTest.java:359)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeAddedTriggerWithAddReplicaPreferredOp_2Shard

Error Message:
Timed out waiting for replicas of new collection to be active null Live Nodes: [127.0.0.1:45393_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for replicas of new collection to be active
null
Live Nodes: [127.0.0.1:45393_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([3E9DE8CFC015A7BB:8706D3105349747D]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.nodeAddedTriggerWithAddReplicaPreferredOp(ComputePlanActionTest.java:559)
        at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeAddedTriggerWithAddReplicaPreferredOp_2Shard(ComputePlanActionTest.java:524)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeAddedTriggerWithAddReplicaPreferredOp_2Shard

Error Message:
Timed out waiting for replicas of new collection to be active null Live Nodes: [127.0.0.1:36291_solr, 127.0.0.1:41533_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for replicas of new collection to be active
null
Live Nodes: [127.0.0.1:36291_solr, 127.0.0.1:41533_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([3E9DE8CFC015A7BB:8706D3105349747D]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.nodeAddedTriggerWithAddReplicaPreferredOp(ComputePlanActionTest.java:585)
        at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeAddedTriggerWithAddReplicaPreferredOp_2Shard(ComputePlanActionTest.java:524)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeLostTriggerWithDeleteNodePreferredOp

Error Message:
Timed out waiting for replicas of new collection to be active null Live Nodes: [127.0.0.1:33929_solr, 127.0.0.1:41899_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for replicas of new collection to be active
null
Live Nodes: [127.0.0.1:33929_solr, 127.0.0.1:41899_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([3E9DE8CFC015A7BB:A1324C2CB7AD3ED2]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeLostTriggerWithDeleteNodePreferredOp(ComputePlanActionTest.java:655)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeAddedTriggerWithAddReplicaPreferredOp_2Shard

Error Message:
Timed out waiting for replicas of new collection to be active null Live Nodes: [127.0.0.1:34851_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for replicas of new collection to be active
null
Live Nodes: [127.0.0.1:34851_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([3E9DE8CFC015A7BB:8706D3105349747D]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.nodeAddedTriggerWithAddReplicaPreferredOp(ComputePlanActionTest.java:559)
        at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeAddedTriggerWithAddReplicaPreferredOp_2Shard(ComputePlanActionTest.java:524)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.handler.component.DistributedMLTComponentTest.test

Error Message:
Error from server at https://127.0.0.1:34333/_zrj/z/collection1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://[ff01::114]:33332/_zrj/z, https://[ff01::083]:33332/_zrj/z, https://[ff01::213]:33332/_zrj/z]

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:34333/_zrj/z/collection1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://[ff01::114]:33332/_zrj/z, https://[ff01::083]:33332/_zrj/z, https://[ff01::213]:33332/_zrj/z]
        at __randomizedtesting.SeedInfo.seed([3E9DE8CFC015A7BB:B6C9D7156EE9CA43]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase.queryServer(BaseDistributedSearchTestCase.java:559)
        at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:611)
        at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:589)
        at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:568)
        at org.apache.solr.handler.component.DistributedMLTComponentTest.test(DistributedMLTComponentTest.java:148)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.handler.component.DistributedMLTComponentTest.test

Error Message:
Error from server at https://127.0.0.1:41829/_zrj/z/collection1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:35665/_zrj/z/collection1, https://[ff01::114]:33332/_zrj/z, https://[ff01::213]:33332/_zrj/z, https://[ff01::083]:33332/_zrj/z]

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:41829/_zrj/z/collection1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:35665/_zrj/z/collection1, https://[ff01::114]:33332/_zrj/z, https://[ff01::213]:33332/_zrj/z, https://[ff01::083]:33332/_zrj/z]
        at __randomizedtesting.SeedInfo.seed([3E9DE8CFC015A7BB:B6C9D7156EE9CA43]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase.queryServer(BaseDistributedSearchTestCase.java:559)
        at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:611)
        at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:589)
        at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:568)
        at org.apache.solr.handler.component.DistributedMLTComponentTest.test(DistributedMLTComponentTest.java:153)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.handler.component.DistributedMLTComponentTest.test

Error Message:
Error from server at https://127.0.0.1:43599/_zrj/z/collection1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://[ff01::114]:33332/_zrj/z, https://[ff01::083]:33332/_zrj/z, https://[ff01::213]:33332/_zrj/z, https://127.0.0.1:42941/_zrj/z/collection1]

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:43599/_zrj/z/collection1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://[ff01::114]:33332/_zrj/z, https://[ff01::083]:33332/_zrj/z, https://[ff01::213]:33332/_zrj/z, https://127.0.0.1:42941/_zrj/z/collection1]
        at __randomizedtesting.SeedInfo.seed([3E9DE8CFC015A7BB:B6C9D7156EE9CA43]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase.queryServer(BaseDistributedSearchTestCase.java:559)
        at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:611)
        at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:589)
        at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:568)
        at org.apache.solr.handler.component.DistributedMLTComponentTest.test(DistributedMLTComponentTest.java:148)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.handler.component.DistributedMLTComponentTest.test

Error Message:
Error from server at https://127.0.0.1:38825/_zrj/z/collection1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:43145/_zrj/z/collection1, https://[ff01::114]:33332/_zrj/z, https://[ff01::083]:33332/_zrj/z, https://[ff01::213]:33332/_zrj/z]

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:38825/_zrj/z/collection1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:43145/_zrj/z/collection1, https://[ff01::114]:33332/_zrj/z, https://[ff01::083]:33332/_zrj/z, https://[ff01::213]:33332/_zrj/z]
        at __randomizedtesting.SeedInfo.seed([3E9DE8CFC015A7BB:B6C9D7156EE9CA43]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase.queryServer(BaseDistributedSearchTestCase.java:559)
        at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:611)
        at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:589)
        at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:568)
        at org.apache.solr.handler.component.DistributedMLTComponentTest.test(DistributedMLTComponentTest.java:148)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeWithMultipleReplicasLost

Error Message:
Timed out waiting for replicas of new collection to be active null Live Nodes: [127.0.0.1:35475_solr, 127.0.0.1:37955_solr, 127.0.0.1:42473_solr, 127.0.0.1:45393_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for replicas of new collection to be active
null
Live Nodes: [127.0.0.1:35475_solr, 127.0.0.1:37955_solr, 127.0.0.1:42473_solr, 127.0.0.1:45393_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([3E9DE8CFC015A7BB:E5D094D486746E7]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeWithMultipleReplicasLost(ComputePlanActionTes

[...truncated too long message...]

riggers Shutting down action executor now
   [junit4]   2> 427251 DEBUG (jetty-closer-386-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 427251 DEBUG (jetty-closer-386-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 427251 DEBUG (jetty-closer-386-thread-1) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 427251 DEBUG (jetty-closer-386-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 427255 INFO  (jetty-closer-386-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@69c40711{/solr,null,UNAVAILABLE}
   [junit4]   2> 427255 INFO  (jetty-closer-386-thread-1) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 427255 ERROR (SUITE-ComputePlanActionTest-seed#[3E9DE8CFC015A7BB]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 427255 INFO  (SUITE-ComputePlanActionTest-seed#[3E9DE8CFC015A7BB]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36463 36463
   [junit4]   2> 427355 INFO  (Thread-235) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36463 36463
   [junit4]   2> 427356 WARN  (Thread-235) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 34 /solr/aliases.json
   [junit4]   2> 16 /solr/configs/conf
   [junit4]   2> 14 /solr/collections/testNodeWithMultipleReplicasLost/terms/shard1
   [junit4]   2> 13 /solr/security.json
   [junit4]   2> 8 /solr/collections/testNodeWithMultipleReplicasLost/terms/shard2
   [junit4]   2> 7 /solr/collections/testNodeAddedTriggerWithAddReplicaPreferredOp_1Shard_4/terms/shard1
   [junit4]   2> 6 /solr/collections/testNodeLost/terms/shard1
   [junit4]   2> 6 /solr/collections/testNodeAddedTriggerWithAddReplicaPreferredOp_1Shard_1/terms/shard1
   [junit4]   2> 6 /solr/collections/testNodeAddedTriggerWithAddReplicaPreferredOp_1Shard_3/terms/shard1
   [junit4]   2> 6 /solr/collections/testSelected1/terms/shard1
   [junit4]   2> 6 /solr/collections/testSelected3/terms/shard1
   [junit4]   2> 6 /solr/collections/testNodeLostTriggerWithDeleteNodePreferredOp_0/terms/shard1
   [junit4]   2> 6 /solr/collections/testNodeLostTriggerWithDeleteNodePreferredOp_0/terms/shard2
   [junit4]   2> 6 /solr/collections/testNodeAddedTriggerWithAddReplicaPreferredOp_1Shard_2/terms/shard1
   [junit4]   2> 5 /solr/collections/testSelected1/terms/shard2
   [junit4]   2> 5 /solr/collections/testSelected2/terms/shard1
   [junit4]   2> 5 /solr/collections/testSelected2/terms/shard2
   [junit4]   2> 5 /solr/collections/testSelected3/terms/shard2
   [junit4]   2> 3 /solr/collections/testNodeAddedTriggerWithAddReplicaPreferredOp_2Shard_0/terms/shard1
   [junit4]   2> 3 /solr/collections/testNodeAdded/terms/shard1
   [junit4]   2> 3 /solr/collections/testNodeAddedTriggerWithAddReplicaPreferredOp_1Shard_0/terms/shard1
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 34 /solr/clusterprops.json
   [junit4]   2> 34 /solr/clusterstate.json
   [junit4]   2> 28 /solr/aliases.json
   [junit4]   2> 9 /solr/overseer_elect/election/72215666382864387-127.0.0.1:45393_solr-n_0000000000
   [junit4]   2> 7 /solr/collections/testNodeLostTriggerWithDeleteNodePreferredOp_0/state.json
   [junit4]   2> 5 /solr/collections/testSelected1/state.json
   [junit4]   2> 5 /solr/collections/testSelected2/state.json
   [junit4]   2> 5 /solr/collections/testNodeWithMultipleReplicasLost/state.json
   [junit4]   2> 5 /solr/collections/testSelected3/state.json
   [junit4]   2> 3 /solr/collections/testNodeAddedTriggerWithAddReplicaPreferredOp_1Shard_4/state.json
   [junit4]   2> 3 /solr/collections/testNodeAddedTriggerWithAddReplicaPreferredOp_1Shard_1/state.json
   [junit4]   2> 3 /solr/collections/testNodeAdded/state.json
   [junit4]   2> 3 /solr/collections/testNodeLost/state.json
   [junit4]   2> 3 /solr/collections/testNodeAddedTriggerWithAddReplicaPreferredOp_1Shard_3/state.json
   [junit4]   2> 3 /solr/collections/testNodeAddedTriggerWithAddReplicaPreferredOp_1Shard_2/state.json
   [junit4]   2> 2 /solr/collections/testNodeAddedTriggerWithAddReplicaPreferredOp_1Shard_0/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 133 /solr/collections
   [junit4]   2> 114 /solr/live_nodes
   [junit4]   2> 2 /solr/autoscaling/events/node_lost_trigger
   [junit4]   2> 2 /solr/autoscaling/events/node_added_trigger
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ComputePlanActionTest_3E9DE8CFC015A7BB-002
   [junit4]   2> Sep 13, 2018 5:29:12 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1891, maxMBSortInHeap=7.315435462371426, sim=RandomSimilarity(queryNorm=false): {}, locale=dyo, timezone=Indian/Antananarivo
   [junit4]   2> NOTE: Linux 4.15.0-32-generic amd64/Oracle Corporation 11 (64-bit)/cpus=8,threads=1,free=157430520,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [TestSkipOverseerOperations, ComputePlanActionTest, ComputePlanActionTest]
   [junit4] Completed [15/15 (9!)] on J2 in 238.57s, 7 tests, 2 failures <<< FAILURES!
   [junit4]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/temp/junit4-J2-20180913_052204_9959329006235767522641.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----
   [junit4]
   [junit4] Tests with failures [seed: 3E9DE8CFC015A7BB] (first 10 out of 13):
   [junit4]   - org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeAddedTriggerWithAddReplicaPreferredOp_2Shard
   [junit4]   - org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeAddedTriggerWithAddReplicaPreferredOp_2Shard
   [junit4]   - org.apache.solr.cloud.TestSkipOverseerOperations.testSkipDownOperations
   [junit4]   - org.apache.solr.handler.component.DistributedMLTComponentTest.test
   [junit4]   - org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeLostTriggerWithDeleteNodePreferredOp
   [junit4]   - org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeLostTriggerWithDeleteNodePreferredOp
   [junit4]   - org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeAddedTriggerWithAddReplicaPreferredOp_2Shard
   [junit4]   - org.apache.solr.handler.component.DistributedMLTComponentTest.test
   [junit4]   - org.apache.solr.handler.component.DistributedMLTComponentTest.test
   [junit4]   - org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeWithMultipleReplicasLost
   [junit4]
   [junit4]
   [junit4] JVM J0:     0.71 ..   381.92 =   381.21s
   [junit4] JVM J1:     0.70 ..   321.08 =   320.38s
   [junit4] JVM J2:     0.71 ..   428.99 =   428.27s
   [junit4] Execution time total: 7 minutes 9 seconds
   [junit4] Tests summary: 15 suites, 50 tests, 3 errors, 10 failures

BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/common-build.xml:1567: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/common-build.xml:1092: There were test failures: 15 suites, 50 tests, 3 errors, 10 failures [seed: 3E9DE8CFC015A7BB]

Total time: 7 minutes 10 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   1/5 failed: org.apache.solr.client.solrj.io.stream.MathExpressionTest
[repro]   1/5 failed: org.apache.solr.cloud.TestSkipOverseerOperations
[repro]   3/5 failed: org.apache.solr.handler.component.DistributedMLTComponentTest
[repro]   5/5 failed: org.apache.solr.cloud.autoscaling.ComputePlanActionTest
[repro]   5/5 failed: org.apache.solr.handler.extraction.ExtractingRequestHandlerTest
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=136f0fee5b1c27047a14c2aa9fc89c23eb69fa08, workspace=/var/lib/jenkins/workspace/Lucene-Solr-7.x-Linux
[WARNINGS] Computing warning deltas based on reference build #2734
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk-10.0.1) - Build # 2736 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/2736/
Java: 64bit/jdk-10.0.1 -XX:-UseCompressedOops -XX:+UseSerialGC

4 tests failed.
FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:38355/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38355/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([C22EB685876604C5:4C22D6F5F27E80EF]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:42281/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42281/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([C22EB685876604C5:4C22D6F5F27E80EF]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:40459/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:40459/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([C22EB685876604C5:4C22D6F5F27E80EF]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:39463/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39463/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([C22EB685876604C5:4C22D6F5F27E80EF]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at jdk.internal.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 12984 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestPullReplicaErrorHandling
   [junit4]   2> 28912 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/init-core-data-001
   [junit4]   2> 28913 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 28914 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl="https://issues.apache.org/jira/browse/SOLR-5776")
   [junit4]   2> 28914 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/tempDir-001
   [junit4]   2> 28914 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 28915 INFO  (Thread-53) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 28915 INFO  (Thread-53) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 28925 ERROR (Thread-53) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 29015 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:37551
   [junit4]   2> 29041 INFO  (zkConnectionManagerCallback-77-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29064 INFO  (jetty-launcher-74-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10.0.1+10
   [junit4]   2> 29082 INFO  (jetty-launcher-74-thread-3) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10.0.1+10
   [junit4]   2> 29082 INFO  (jetty-launcher-74-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10.0.1+10
   [junit4]   2> 29083 INFO  (jetty-launcher-74-thread-4) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10.0.1+10
   [junit4]   2> 29249 INFO  (jetty-launcher-74-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 29249 INFO  (jetty-launcher-74-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 29250 INFO  (jetty-launcher-74-thread-2) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 29250 INFO  (jetty-launcher-74-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2b17399{/solr,null,AVAILABLE}
   [junit4]   2> 29255 INFO  (jetty-launcher-74-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@45efd000{HTTP/1.1,[http/1.1]}{127.0.0.1:44983}
   [junit4]   2> 29255 INFO  (jetty-launcher-74-thread-2) [    ] o.e.j.s.Server Started @29280ms
   [junit4]   2> 29255 INFO  (jetty-launcher-74-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44983}
   [junit4]   2> 29255 ERROR (jetty-launcher-74-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 29255 INFO  (jetty-launcher-74-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 29256 INFO  (jetty-launcher-74-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.6.0
   [junit4]   2> 29256 INFO  (jetty-launcher-74-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 29256 INFO  (jetty-launcher-74-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 29256 INFO  (jetty-launcher-74-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T09:11:05.990176Z
   [junit4]   2> 29259 INFO  (jetty-launcher-74-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 29259 INFO  (jetty-launcher-74-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 29259 INFO  (jetty-launcher-74-thread-3) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 29260 INFO  (jetty-launcher-74-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1dcd20f2{/solr,null,AVAILABLE}
   [junit4]   2> 29261 INFO  (jetty-launcher-74-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@76e76d63{HTTP/1.1,[http/1.1]}{127.0.0.1:35337}
   [junit4]   2> 29261 INFO  (jetty-launcher-74-thread-3) [    ] o.e.j.s.Server Started @29286ms
   [junit4]   2> 29261 INFO  (jetty-launcher-74-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35337}
   [junit4]   2> 29262 ERROR (jetty-launcher-74-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 29262 INFO  (jetty-launcher-74-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 29262 INFO  (jetty-launcher-74-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.6.0
   [junit4]   2> 29262 INFO  (jetty-launcher-74-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 29262 INFO  (jetty-launcher-74-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 29262 INFO  (jetty-launcher-74-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T09:11:05.996701Z
   [junit4]   2> 29273 INFO  (jetty-launcher-74-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 29274 INFO  (jetty-launcher-74-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 29274 INFO  (jetty-launcher-74-thread-4) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 29291 INFO  (jetty-launcher-74-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@43b815cd{/solr,null,AVAILABLE}
   [junit4]   2> 29292 INFO  (jetty-launcher-74-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2780c7de{HTTP/1.1,[http/1.1]}{127.0.0.1:43995}
   [junit4]   2> 29292 INFO  (jetty-launcher-74-thread-4) [    ] o.e.j.s.Server Started @29317ms
   [junit4]   2> 29292 INFO  (jetty-launcher-74-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43995}
   [junit4]   2> 29292 ERROR (jetty-launcher-74-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 29292 INFO  (jetty-launcher-74-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 29292 INFO  (jetty-launcher-74-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.6.0
   [junit4]   2> 29292 INFO  (jetty-launcher-74-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 29293 INFO  (jetty-launcher-74-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 29293 INFO  (jetty-launcher-74-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T09:11:06.027030Z
   [junit4]   2> 29293 INFO  (jetty-launcher-74-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 29293 INFO  (jetty-launcher-74-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 29293 INFO  (jetty-launcher-74-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 29302 INFO  (zkConnectionManagerCallback-81-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29306 INFO  (jetty-launcher-74-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1c7e60e5{/solr,null,AVAILABLE}
   [junit4]   2> 29306 INFO  (jetty-launcher-74-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 29307 INFO  (jetty-launcher-74-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5b8004eb{HTTP/1.1,[http/1.1]}{127.0.0.1:34527}
   [junit4]   2> 29307 INFO  (jetty-launcher-74-thread-1) [    ] o.e.j.s.Server Started @29332ms
   [junit4]   2> 29307 INFO  (jetty-launcher-74-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34527}
   [junit4]   2> 29307 ERROR (jetty-launcher-74-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 29307 INFO  (jetty-launcher-74-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 29307 INFO  (jetty-launcher-74-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.6.0
   [junit4]   2> 29307 INFO  (jetty-launcher-74-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 29307 INFO  (jetty-launcher-74-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 29307 INFO  (jetty-launcher-74-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T09:11:06.041726Z
   [junit4]   2> 29315 INFO  (zkConnectionManagerCallback-83-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29329 INFO  (jetty-launcher-74-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 29343 INFO  (zkConnectionManagerCallback-79-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29344 INFO  (jetty-launcher-74-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 29377 INFO  (zkConnectionManagerCallback-85-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29382 INFO  (jetty-launcher-74-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 29554 INFO  (jetty-launcher-74-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37551/solr
   [junit4]   2> 29598 INFO  (jetty-launcher-74-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37551/solr
   [junit4]   2> 29618 INFO  (zkConnectionManagerCallback-90-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29626 INFO  (zkConnectionManagerCallback-93-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29637 INFO  (zkConnectionManagerCallback-95-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29649 INFO  (zkConnectionManagerCallback-97-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29694 INFO  (jetty-launcher-74-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37551/solr
   [junit4]   2> 29777 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43995_solr
   [junit4]   2> 29777 INFO  (zkConnectionManagerCallback-105-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29779 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.c.Overseer Overseer (id=72216554442522631-127.0.0.1:43995_solr-n_0000000000) starting
   [junit4]   2> 29798 INFO  (jetty-launcher-74-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37551/solr
   [junit4]   2> 29842 INFO  (jetty-launcher-74-thread-3) [n:127.0.0.1:35337_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35337_solr
   [junit4]   2> 29842 INFO  (zkConnectionManagerCallback-109-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29917 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 29921 INFO  (zkCallback-96-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 29933 INFO  (zkConnectionManagerCallback-111-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29953 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 29957 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 29957 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34527_solr
   [junit4]   2> 29970 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 29973 INFO  (zkCallback-96-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 30030 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 30033 INFO  (zkConnectionManagerCallback-119-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30033 INFO  (zkConnectionManagerCallback-114-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30049 INFO  (zkConnectionManagerCallback-123-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30058 INFO  (jetty-launcher-74-thread-3) [n:127.0.0.1:35337_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 30060 INFO  (jetty-launcher-74-thread-3) [n:127.0.0.1:35337_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37551/solr ready
   [junit4]   2> 30064 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 30065 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37551/solr ready
   [junit4]   2> 30066 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 30073 INFO  (zkConnectionManagerCallback-130-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30079 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 30079 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44983_solr
   [junit4]   2> 30084 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 30084 INFO  (zkCallback-96-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 30084 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 30085 INFO  (OverseerStateUpdate-72216554442522631-127.0.0.1:43995_solr-n_0000000000) [n:127.0.0.1:43995_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:43995_solr
   [junit4]   2> 30093 INFO  (jetty-launcher-74-thread-3) [n:127.0.0.1:35337_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 30103 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 30103 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43995_solr
   [junit4]   2> 30109 INFO  (zkCallback-118-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 30122 INFO  (jetty-launcher-74-thread-3) [n:127.0.0.1:35337_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35337.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30150 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 30150 INFO  (zkCallback-96-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 30150 INFO  (zkCallback-118-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 30151 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 30151 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 30152 INFO  (jetty-launcher-74-thread-3) [n:127.0.0.1:35337_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35337.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30152 INFO  (jetty-launcher-74-thread-3) [n:127.0.0.1:35337_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35337.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30153 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37551/solr ready
   [junit4]   2> 30153 INFO  (jetty-launcher-74-thread-3) [n:127.0.0.1:35337_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/tempDir-001/node3/.
   [junit4]   2> 30173 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 30214 INFO  (zkCallback-113-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
   [junit4]   2> 30214 INFO  (zkCallback-122-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 30233 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43995.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30241 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43995.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30241 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43995.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30245 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 30249 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/tempDir-001/node4/.
   [junit4]   2> 30259 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34527.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30266 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34527.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30266 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34527.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30268 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/tempDir-001/node1/.
   [junit4]   2> 30305 INFO  (zkConnectionManagerCallback-138-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30314 INFO  (zkCallback-122-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 30369 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 30373 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37551/solr ready
   [junit4]   2> 30417 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 30434 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44983.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30441 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44983.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30442 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44983.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30443 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/tempDir-001/node2/.
   [junit4]   2> 30575 INFO  (zkConnectionManagerCallback-143-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30645 INFO  (zkConnectionManagerCallback-148-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30647 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 30647 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37551/solr ready
   [junit4]   2> 30654 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@76e76d63{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 30662 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=665933534
   [junit4]   2> 30662 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 30662 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@62778794: rootName = solr_35337, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@1ebc0f06
   [junit4]   2> 30670 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 30670 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@74bc4a0c: rootName = solr_35337, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@66ed5911
   [junit4]   2> 30675 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 30675 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@75b24664: rootName = solr_35337, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6b576573
   [junit4]   2> 30680 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:35337_solr
   [junit4]   2> 30681 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30681 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30681 INFO  (zkCallback-122-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30681 INFO  (zkCallback-118-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30681 INFO  (zkCallback-113-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30682 INFO  (zkCallback-96-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30683 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 30698 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 30731 INFO  (zkCallback-147-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30731 INFO  (zkCallback-129-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30735 INFO  (zkCallback-137-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30735 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1dcd20f2{/solr,null,UNAVAILABLE}
   [junit4]   2> 30735 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 30736 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10.0.1+10
   [junit4]   2> 30864 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 30864 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 30864 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 30870 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@215ebd2f{/solr,null,AVAILABLE}
   [junit4]   2> 30886 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@f8a8afc{HTTP/1.1,[http/1.1]}{127.0.0.1:33583}
   [junit4]   2> 30886 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.Server Started @30911ms
   [junit4]   2> 30886 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39463}
   [junit4]   2> 30887 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 30887 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 30887 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.6.0
   [junit4]   2> 30887 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 30887 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 30887 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T09:11:07.621582Z
   [junit4]   2> 30933 INFO  (zkConnectionManagerCallback-150-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30935 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 31446 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37551/solr
   [junit4]   2> 31462 INFO  (zkConnectionManagerCallback-154-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 31478 INFO  (zkConnectionManagerCallback-156-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 31484 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 31488 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 31488 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39463_solr
   [junit4]   2> 31500 INFO  (zkCallback-113-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 31500 INFO  (zkCallback-137-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 31500 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 31500 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 31500 INFO  (zkCallback-147-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 31500 INFO  (zkCallback-129-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 31502 INFO  (zkCallback-118-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 31512 INFO  (zkCallback-155-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 31587 INFO  (zkConnectionManagerCallback-163-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 31592 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 31593 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37551/solr ready
   [junit4]   2> 31617 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 31646 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39463.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 31672 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39463.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 31673 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39463.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 31678 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/tempDir-001/node3/.
   [junit4]   2> 31787 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:33583/solr. Proxy: http://127.0.0.1:39463/solr
   [junit4]   2> 31790 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@2780c7de{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 31791 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=211244492
   [junit4]   2> 31791 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 31791 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@37e3d6a7: rootName = solr_43995, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@65c4d10e
   [junit4]   2> 31799 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 31799 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@738a3b72: rootName = solr_43995, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@66ed5911
   [junit4]   2> 31803 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 31804 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6038370: rootName = solr_43995, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6b576573
   [junit4]   2> 31808 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:43995_solr
   [junit4]   2> 31809 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31809 INFO  (zkCallback-118-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31809 INFO  (zkCallback-129-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31809 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31809 INFO  (zkCallback-137-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31809 INFO  (zkCallback-113-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31809 INFO  (zkCallback-155-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31810 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 31814 INFO  (zkCallback-147-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31818 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.Overseer Overseer (id=72216554442522631-127.0.0.1:43995_solr-n_0000000000) closing
   [junit4]   2> 31818 INFO  (OverseerStateUpdate-72216554442522631-127.0.0.1:43995_solr-n_0000000000) [n:127.0.0.1:43995_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:43995_solr
   [junit4]   2> 31838 WARN  (OverseerAutoScalingTriggerThread-72216554442522631-127.0.0.1:43995_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 31839 INFO  (zkCallback-162-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31848 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@43b815cd{/solr,null,UNAVAILABLE}
   [junit4]   2> 31848 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 31849 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34527_solr
   [junit4]   2> 31850 INFO  (zkCallback-110-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.c.Overseer Overseer (id=72216554442522635-127.0.0.1:34527_solr-n_0000000002) starting
   [junit4]   2> 31850 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10.0.1+10
   [junit4]   2> 31925 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 31925 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 31925 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 31938 INFO  (OverseerStateUpdate-72216554442522635-127.0.0.1:34527_solr-n_0000000002) [n:127.0.0.1:34527_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:34527_solr
   [junit4]   2> 31965 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2b8b4b60{/solr,null,AVAILABLE}
   [junit4]   2> 31966 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3e1abbc{HTTP/1.1,[http/1.1]}{127.0.0.1:45319}
   [junit4]   2> 31966 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.Server Started @31991ms
   [junit4]   2> 31966 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41137}
   [junit4]   2> 31966 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 31978 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 31978 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.6.0
   [junit4]   2> 31978 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 31978 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 31978 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T09:11:08.712531Z
   [junit4]   2> 32069 INFO  (zkConnectionManagerCallback-166-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 32071 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 32205 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37551/solr
   [junit4]   2> 32212 INFO  (zkConnectionManagerCallback-170-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 32216 INFO  (zkConnectionManagerCallback-172-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 32220 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 32223 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 32223 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41137_solr
   [junit4]   2> 32224 INFO  (zkCallback-147-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32224 INFO  (zkCallback-113-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32224 INFO  (zkCallback-129-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32224 INFO  (zkCallback-155-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32224 INFO  (zkCallback-137-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32224 INFO  (zkCallback-162-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32224 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32224 INFO  (zkCallback-171-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32253 INFO  (zkConnectionManagerCallback-179-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 32255 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 32259 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37551/solr ready
   [junit4]   2> 32259 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 32271 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41137.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 32277 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41137.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 32277 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41137.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 32278 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/tempDir-001/node4/.
   [junit4]   2> 32333 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:45319/solr. Proxy: http://127.0.0.1:41137/solr
   [junit4]   2> 32334 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@5b8004eb{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 32334 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1909036678
   [junit4]   2> 32335 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 32335 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2ad30f93: rootName = solr_34527, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@2e00451d
   [junit4]   2> 32342 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 32342 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@27482a76: rootName = solr_34527, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@66ed5911
   [junit4]   2> 32345 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 32345 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1f360f5b: rootName = solr_34527, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6b576573
   [junit4]   2> 32346 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:34527_solr
   [junit4]   2> 32347 INFO  (zkCallback-155-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32347 INFO  (zkCallback-147-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32347 INFO  (zkCallback-162-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32347 INFO  (zkCallback-113-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32347 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 32348 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32350 INFO  (zkCallback-171-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32355 INFO  (zkCallback-178-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32358 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.Overseer Overseer (id=72216554442522635-127.0.0.1:34527_solr-n_0000000002) closing
   [junit4]   2> 32358 INFO  (OverseerStateUpdate-72216554442522635-127.0.0.1:34527_solr-n_0000000002) [n:127.0.0.1:34527_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:34527_solr
   [junit4]   2> 32358 INFO  (zkCallback-129-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32358 WARN  (OverseerAutoScalingTriggerThread-72216554442522635-127.0.0.1:34527_solr-n_0000000002) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 32362 INFO  (zkCallback-137-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32389 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1c7e60e5{/solr,null,UNAVAILABLE}
   [junit4]   2> 32389 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 32389 INFO  (zkCallback-113-thread-2) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44983_solr
   [junit4]   2> 32390 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10.0.1+10
   [junit4]   2> 32390 INFO  (zkCallback-113-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.c.Overseer Overseer (id=72216554442522637-127.0.0.1:44983_solr-n_0000000003) starting
   [junit4]   2> 32424 INFO  (OverseerStateUpdate-72216554442522637-127.0.0.1:44983_solr-n_0000000003) [n:127.0.0.1:44983_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:44983_solr
   [junit4]   2> 32435 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 32435 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 32435 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 32435 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@14323a68{/solr,null,AVAILABLE}
   [junit4]   2> 32436 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@73585f83{HTTP/1.1,[http/1.1]}{127.0.0.1:32851}
   [junit4]   2> 32436 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.Server Started @32461ms
   [junit4]   2> 32436 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41139}
   [junit4]   2> 32436 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 32437 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 32437 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.6.0
   [junit4]   2> 32437 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 32437 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 32437 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T09:11:09.171354Z
   [junit4]   2> 32456 INFO  (zkConnectionManagerCallback-182-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 32457 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 32622 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37551/solr
   [junit4]   2> 32641 INFO  (zkConnectionManagerCallback-186-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 32665 INFO  (zkConnectionManagerCallback-188-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 32681 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 32684 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 32684 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41139_solr
   [junit4]   2> 32694 INFO  (zkCallback-147-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32694 INFO  (zkCallback-162-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32694 INFO  (zkCallback-113-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32694 INFO  (zkCallback-178-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32694 INFO  (zkCallback-171-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32694 INFO  (zkCallback-155-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32695 INFO  (zkCallback-137-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32702 INFO  (zkCallback-187-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32736 INFO  (zkConnectionManagerCallback-195-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 32754 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 32760 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37551/solr ready
   [junit4]   2> 32786 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 32801 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41139.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 32809 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41139.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 32809 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41139.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 32810 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/tempDir-001/node1/.
   [junit4]   2> 32915 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:32851/solr. Proxy: http://127.0.0.1:41139/solr
   [junit4]   2> 32919 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@45efd000{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 32924 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2118852418
   [junit4]   2> 32924 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 32924 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2efc7c93: rootName = solr_44983, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@3f1b951
   [junit4]   2> 32931 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 32931 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@60a81a5b: rootName = solr_44983, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@66ed5911
   [junit4]   2> 32935 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 32935 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@36a51dfc: rootName = solr_44983, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6b576573
   [junit4]   2> 32938 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:44983_solr
   [junit4]   2> 32939 INFO  (zkCallback-162-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32939 INFO  (zkCallback-113-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32939 INFO  (zkCallback-147-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32940 INFO  (zkCallback-155-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32940 INFO  (zkCallback-187-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32940 INFO  (zkCallback-171-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32940 INFO  (zkCallback-137-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32939 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 32941 INFO  (zkCallback-178-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32948 INFO  (zkCallback-194-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32950 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.Overseer Overseer (id=72216554442522637-127.0.0.1:44983_solr-n_0000000003) closing
   [junit4]   2> 32950 INFO  (OverseerStateUpdate-72216554442522637-127.0.0.1:44983_solr-n_0000000003) [n:127.0.0.1:44983_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:44983_solr
   [junit4]   2> 32961 WARN  (OverseerAutoScalingTriggerThread-72216554442522637-127.0.0.1:44983_solr-n_0000000003) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 32975 INFO  (zkCallback-155-thread-1) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39463_solr
   [junit4]   2> 32976 INFO  (zkCallback-155-thread-1) [n:127.0.0.1:39463_solr    ] o.a.s.c.Overseer Overseer (id=72216554442522645-127.0.0.1:39463_solr-n_0000000004) starting
   [junit4]   2> 32982 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2b17399{/solr,null,UNAVAILABLE}
   [junit4]   2> 32982 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 32984 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10.0.1+10
   [junit4]   2> 33001 INFO  (OverseerStateUpdate-72216554442522645-127.0.0.1:39463_solr-n_0000000004) [n:127.0.0.1:39463_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:39463_solr
   [junit4]   2> 33033 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 33033 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 33034 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 33038 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7e53be35{/solr,null,AVAILABLE}
   [junit4]   2> 33038 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7861210c{HTTP/1.1,[http/1.1]}{127.0.0.1:36203}
   [junit4]   2> 33039 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.Server Started @33063ms
   [junit4]   2> 33039 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40889}
   [junit4]   2> 33040 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 33057 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 33057 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.6.0
   [junit4]   2> 33057 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 33057 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 33057 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T09:11:09.791414Z
   [junit4]   2> 33077 INFO  (zkConnectionManagerCallback-198-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 33078 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 33387 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37551/solr
   [junit4]   2> 33421 INFO  (zkConnectionManagerCallback-202-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 33447 INFO  (zkConnectionManagerCallback-204-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 33454 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 33458 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 33458 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40889_solr
   [junit4]   2> 33459 INFO  (zkCallback-162-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 33459 INFO  (zkCallback-147-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 33459 INFO  (zkCallback-194-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 33459 INFO  (zkCallback-178-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 33460 INFO  (zkCallback-187-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 33460 INFO  (zkCallback-171-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 33460 INFO  (zkCallback-155-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 33471 INFO  (zkCallback-203-thread-1) [    ] o.a

[...truncated too long message...]

ader Updated live nodes from ZooKeeper... (3) -> (1)
   [junit4]   2> 1265002 INFO  (zkCallback-239-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (1)
   [junit4]   2> 1265002 INFO  (zkCallback-223-thread-6) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (1)
   [junit4]   2> 1265002 INFO  (jetty-closer-143-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1265002 INFO  (zkCallback-271-thread-6) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (1)
   [junit4]   2> 1265003 INFO  (jetty-closer-143-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@31c1aad9: rootName = solr_40459, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@658f2ef2
   [junit4]   2> 1265004 INFO  (jetty-closer-143-thread-2) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:40459_solr
   [junit4]   2> 1265006 INFO  (jetty-closer-143-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1265006 INFO  (jetty-closer-143-thread-1) [    ] o.a.s.c.Overseer Overseer (id=72216880531701781-127.0.0.1:43661_solr-n_0000000004) closing
   [junit4]   2> 1265006 INFO  (OverseerStateUpdate-72216880531701781-127.0.0.1:43661_solr-n_0000000004) [n:127.0.0.1:43661_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:43661_solr
   [junit4]   2> 1265006 INFO  (zkCallback-230-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1265007 WARN  (OverseerAutoScalingTriggerThread-72216880531701781-127.0.0.1:43661_solr-n_0000000004) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1265007 INFO  (zkCallback-246-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1265007 INFO  (zkCallback-271-thread-6) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1265007 INFO  (zkCallback-255-thread-3) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1265007 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10090dec6e1001e, likely client has closed socket
   [junit4]   2> 1265007 INFO  (zkCallback-223-thread-6) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1265007 INFO  (zkCallback-239-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1265008 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10090dec6e10016, likely client has closed socket
   [junit4]   2> 1265008 INFO  (jetty-closer-143-thread-4) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@23eca0b5{/solr,null,UNAVAILABLE}
   [junit4]   2> 1265008 INFO  (jetty-closer-143-thread-4) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1265009 INFO  (zkCallback-239-thread-2) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40459_solr
   [junit4]   2> 1265009 INFO  (jetty-closer-143-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@cf361ed{/solr,null,UNAVAILABLE}
   [junit4]   2> 1265009 INFO  (jetty-closer-143-thread-1) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1265013 INFO  (jetty-closer-143-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2fbb5a85{/solr,null,UNAVAILABLE}
   [junit4]   2> 1265013 INFO  (zkCallback-255-thread-3) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33301_solr
   [junit4]   2> 1265014 INFO  (jetty-closer-143-thread-2) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1266506 WARN  (zkCallback-262-thread-2) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1266507 INFO  (jetty-closer-143-thread-3) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@58fb984d{/solr,null,UNAVAILABLE}
   [junit4]   2> 1266507 INFO  (jetty-closer-143-thread-3) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1266509 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1266509 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:39349 39349
   [junit4]   2> 1266616 INFO  (Thread-105) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:39349 39349
   [junit4]   2> 1266617 WARN  (Thread-105) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 18 /solr/aliases.json
   [junit4]   2> 17 /solr/clusterprops.json
   [junit4]   2> 9 /solr/security.json
   [junit4]   2> 8 /solr/collections/pull_replica_error_handling_test_pull_replica_disconnects_from_zoo_keeper/terms/shard1
   [junit4]   2> 8 /solr/configs/conf
   [junit4]   2> 8 /solr/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/terms/shard2
   [junit4]   2> 7 /solr/collections/pull_replica_error_handling_test_cant_connect_to_leader/terms/shard1
   [junit4]   2> 7 /solr/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/terms/shard1
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 18 /solr/clusterstate.json
   [junit4]   2> 10 /solr/clusterprops.json
   [junit4]   2> 7 /solr/collections/pull_replica_error_handling_test_pull_replica_disconnects_from_zoo_keeper/state.json
   [junit4]   2> 4 /solr/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/state.json
   [junit4]   2> 4 /solr/overseer_elect/election/72216880531701768-127.0.0.1:40113_solr-n_0000000000
   [junit4]   2> 2 /solr/collections/pull_replica_error_handling_test_cant_connect_to_leader/state.json
   [junit4]   2> 2 /solr/overseer_elect/election/72216880531701789-127.0.0.1:33301_solr-n_0000000006
   [junit4]   2> 2 /solr/autoscaling.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 18 /solr/collections
   [junit4]   2> 17 /solr/live_nodes
   [junit4]   2> 2 /solr/overseer/queue
   [junit4]   2> 2 /solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 2 /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 2 /solr/overseer/queue-work
   [junit4]   2> 2 /solr/overseer/collection-queue-work
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-002
   [junit4]   2> Sep 13, 2018 10:44:38 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), sim=RandomSimilarity(queryNorm=false): {}, locale=fr-BJ, timezone=Australia/Eucla
   [junit4]   2> NOTE: Linux 4.15.0-32-generic amd64/Oracle Corporation 10.0.1 (64-bit)/cpus=8,threads=1,free=401631280,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [TestPullReplicaErrorHandling, TestPullReplicaErrorHandling]
   [junit4] Completed [5/5 (5!)] on J1 in 637.28s, 3 tests, 1 error <<< FAILURES!

[...truncated 15 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/common-build.xml:1567: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/common-build.xml:1092: There were test failures: 5 suites, 15 tests, 5 errors [seed: C22EB685876604C5]

Total time: 21 minutes 9 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   5/5 failed: org.apache.solr.cloud.TestPullReplicaErrorHandling
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=f76a424aa2b1a29eda229e0e7b292551d96e9d29, workspace=/var/lib/jenkins/workspace/Lucene-Solr-7.x-Linux
[WARNINGS] Computing warning deltas based on reference build #2735
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk-9.0.4) - Build # 2737 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/2737/
Java: 64bit/jdk-9.0.4 -XX:-UseCompressedOops -XX:+UseParallelGC

4 tests failed.
FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:37573/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37573/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([82E4B50E98F6F15C:CE8D57EEDEE7576]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:33393/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33393/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([82E4B50E98F6F15C:CE8D57EEDEE7576]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:44959/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:44959/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([82E4B50E98F6F15C:CE8D57EEDEE7576]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:38035/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38035/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([82E4B50E98F6F15C:CE8D57EEDEE7576]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at jdk.internal.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 13356 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestPullReplicaErrorHandling
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-001/init-core-data-001
   [junit4]   2> 190633 WARN  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=69 numCloses=69
   [junit4]   2> 190633 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 190634 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl="https://issues.apache.org/jira/browse/SOLR-5776")
   [junit4]   2> 190634 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 190634 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-001/tempDir-001
   [junit4]   2> 190634 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 190634 INFO  (Thread-791) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 190634 INFO  (Thread-791) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 190639 ERROR (Thread-791) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 190734 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:41657
   [junit4]   2> 190736 INFO  (zkConnectionManagerCallback-479-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190738 INFO  (jetty-launcher-476-thread-4) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
   [junit4]   2> 190739 INFO  (jetty-launcher-476-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
   [junit4]   2> 190739 INFO  (jetty-launcher-476-thread-3) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
   [junit4]   2> 190739 INFO  (jetty-launcher-476-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
   [junit4]   2> 190744 INFO  (jetty-launcher-476-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 190744 INFO  (jetty-launcher-476-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-4) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@29df7c26{/solr,null,AVAILABLE}
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-3) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7348b30{/solr,null,AVAILABLE}
   [junit4]   2> 190746 INFO  (jetty-launcher-476-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3457a8e5{/solr,null,AVAILABLE}
   [junit4]   2> 190746 INFO  (jetty-launcher-476-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@715f57a1{/solr,null,AVAILABLE}
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1d89ee02{HTTP/1.1,[http/1.1]}{127.0.0.1:36145}
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@27bcfccc{HTTP/1.1,[http/1.1]}{127.0.0.1:39833}
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-2) [    ] o.e.j.s.Server Started @190784ms
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3c2bbef7{HTTP/1.1,[http/1.1]}{127.0.0.1:46405}
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-4) [    ] o.e.j.s.Server Started @190784ms
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4998eabd{HTTP/1.1,[http/1.1]}{127.0.0.1:42731}
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-1) [    ] o.e.j.s.Server Started @190784ms
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-3) [    ] o.e.j.s.Server Started @190784ms
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36145}
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39833}
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46405}
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42731}
   [junit4]   2> 190748 ERROR (jetty-launcher-476-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 190748 ERROR (jetty-launcher-476-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 190748 ERROR (jetty-launcher-476-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 190748 ERROR (jetty-launcher-476-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.6.0
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.6.0
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.6.0
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.6.0
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T13:57:26.773903Z
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T13:57:26.773919Z
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T13:57:26.773931Z
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T13:57:26.773904Z
   [junit4]   2> 190756 INFO  (zkConnectionManagerCallback-486-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190756 INFO  (jetty-launcher-476-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 190756 INFO  (zkConnectionManagerCallback-484-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190756 INFO  (zkConnectionManagerCallback-485-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190756 INFO  (jetty-launcher-476-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 190756 INFO  (zkConnectionManagerCallback-487-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190756 INFO  (jetty-launcher-476-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 190757 INFO  (jetty-launcher-476-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 190761 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100919903c10002, likely client has closed socket
   [junit4]   2> 190761 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100919903c10003, likely client has closed socket
   [junit4]   2> 190809 INFO  (jetty-launcher-476-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41657/solr
   [junit4]   2> 190826 INFO  (zkConnectionManagerCallback-491-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190839 INFO  (jetty-launcher-476-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41657/solr
   [junit4]   2> 190844 INFO  (zkConnectionManagerCallback-494-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190857 INFO  (zkConnectionManagerCallback-497-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190859 INFO  (jetty-launcher-476-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41657/solr
   [junit4]   2> 190863 INFO  (zkConnectionManagerCallback-505-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190864 INFO  (zkConnectionManagerCallback-502-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190870 INFO  (zkConnectionManagerCallback-507-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190870 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36145_solr
   [junit4]   2> 190871 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.c.Overseer Overseer (id=72217680416931849-127.0.0.1:36145_solr-n_0000000000) starting
   [junit4]   2> 190885 INFO  (jetty-launcher-476-thread-4) [n:127.0.0.1:39833_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39833_solr
   [junit4]   2> 190887 INFO  (zkCallback-501-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 190888 INFO  (zkCallback-506-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 190889 INFO  (zkConnectionManagerCallback-516-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190890 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 190891 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41657/solr ready
   [junit4]   2> 190892 INFO  (jetty-launcher-476-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41657/solr
   [junit4]   2> 190894 INFO  (OverseerStateUpdate-72217680416931849-127.0.0.1:36145_solr-n_0000000000) [n:127.0.0.1:36145_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:36145_solr
   [junit4]   2> 190898 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 190898 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36145_solr
   [junit4]   2> 190904 INFO  (zkConnectionManagerCallback-520-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190905 INFO  (zkCallback-515-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 190907 INFO  (zkCallback-506-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 190907 INFO  (zkCallback-501-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 190909 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 190910 INFO  (zkConnectionManagerCallback-525-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190910 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 190910 INFO  (jetty-launcher-476-thread-4) [n:127.0.0.1:39833_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 190911 INFO  (jetty-launcher-476-thread-4) [n:127.0.0.1:39833_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41657/solr ready
   [junit4]   2> 190911 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 190911 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46405_solr
   [junit4]   2> 190912 INFO  (zkCallback-506-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 190918 INFO  (zkConnectionManagerCallback-529-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190922 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 190922 INFO  (jetty-launcher-476-thread-4) [n:127.0.0.1:39833_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 190923 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36145.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190924 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 190924 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42731_solr
   [junit4]   2> 190925 INFO  (zkCallback-506-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 190928 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36145.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190928 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36145.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190929 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-001/tempDir-001/node2/.
   [junit4]   2> 190929 INFO  (zkCallback-493-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 190930 INFO  (zkCallback-493-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 190931 INFO  (zkCallback-515-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
   [junit4]   2> 190931 INFO  (zkCallback-524-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
   [junit4]   2> 190931 INFO  (zkCallback-501-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
   [junit4]   2> 190931 INFO  (zkCallback-528-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 190933 INFO  (zkConnectionManagerCallback-536-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190934 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 190938 INFO  (jetty-launcher-476-thread-4) [n:127.0.0.1:39833_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39833.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190939 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41657/solr ready
   [junit4]   2> 190944 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 190944 INFO  (jetty-launcher-476-thread-4) [n:127.0.0.1:39833_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39833.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190944 INFO  (zkConnectionManagerCallback-541-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190944 INFO  (jetty-launcher-476-thread-4) [n:127.0.0.1:39833_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39833.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190945 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 190945 INFO  (jetty-launcher-476-thread-4) [n:127.0.0.1:39833_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-001/tempDir-001/node4/.
   [junit4]   2> 190945 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41657/solr ready
   [junit4]   2> 190946 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 190953 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46405.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190955 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42731.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190958 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46405.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190958 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46405.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190959 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-001/tempDir-001/node1/.
   [junit4]   2> 190959 WARN  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.m.r.j.JmxMetricsReporter Unable to register gauge
   [junit4]   2> javax.management.InstanceNotFoundException: solr_42731:dom1=solr,dom2=jvm,name0=memory,name1=pools,name2=CodeHeap-'profiled-nmethods',name=committed
   [junit4]   2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1083) ~[?:?]
   [junit4]   2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:423) ~[?:?]
   [junit4]   2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:411) ~[?:?]
   [junit4]   2> at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546) ~[?:?]
   [junit4]   2> at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.registerMBean(JmxMetricsReporter.java:530) ~[java/:?]
   [junit4]   2> at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.onGaugeAdded(JmxMetricsReporter.java:565) [java/:?]
   [junit4]   2> at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter.lambda$start$0(JmxMetricsReporter.java:738) [java/:?]
   [junit4]   2> at java.util.HashMap.forEach(HashMap.java:1341) [?:?]
   [junit4]   2> at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter.start(JmxMetricsReporter.java:728) [java/:?]
   [junit4]   2> at org.apache.solr.metrics.reporters.SolrJmxReporter.doInit(SolrJmxReporter.java:110) [java/:?]
   [junit4]   2> at org.apache.solr.metrics.SolrMetricReporter.init(SolrMetricReporter.java:70) [java/:?]
   [junit4]   2> at org.apache.solr.metrics.SolrMetricManager.loadReporter(SolrMetricManager.java:941) [java/:?]
   [junit4]   2> at org.apache.solr.metrics.SolrMetricManager.loadReporters(SolrMetricManager.java:868) [java/:?]
   [junit4]   2> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:590) [java/:?]
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:252) [java/:?]
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:172) [java/:?]
   [junit4]   2> at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2> at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) [java/:?]
   [junit4]   2> at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2> at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) [java/:?]
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) [java/:?]
   [junit4]   2> at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) [java/:?]
   [junit4]   2> at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) [java/:?]
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) [java/:?]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) [?:?]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) [?:?]
   [junit4]   2> at java.lang.Thread.run(Thread.java:844) [?:?]
   [junit4]   2> 190968 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42731.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190968 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42731.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190969 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-001/tempDir-001/node3/.
   [junit4]   2> 191012 INFO  (zkConnectionManagerCallback-545-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191014 INFO  (zkConnectionManagerCallback-550-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191015 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 191015 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41657/solr ready
   [junit4]   2> 191018 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@1d89ee02{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 191018 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1843628192
   [junit4]   2> 191019 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 191019 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5d0f8bbd: rootName = solr_36145, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@3304304
   [junit4]   2> 191023 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 191023 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3dfe3db1: rootName = solr_36145, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2e28778b
   [junit4]   2> 191025 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 191025 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5dce3633: rootName = solr_36145, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@44326a59
   [junit4]   2> 191026 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:36145_solr
   [junit4]   2> 191026 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 191026 INFO  (zkCallback-515-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191026 INFO  (zkCallback-528-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191026 INFO  (zkCallback-524-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191027 INFO  (zkCallback-506-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191026 INFO  (zkCallback-501-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191027 INFO  (zkCallback-493-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191027 INFO  (zkCallback-535-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191028 INFO  (zkCallback-540-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191029 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.Overseer Overseer (id=72217680416931849-127.0.0.1:36145_solr-n_0000000000) closing
   [junit4]   2> 191029 INFO  (zkCallback-549-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191029 INFO  (OverseerStateUpdate-72217680416931849-127.0.0.1:36145_solr-n_0000000000) [n:127.0.0.1:36145_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36145_solr
   [junit4]   2> 191029 WARN  (OverseerAutoScalingTriggerThread-72217680416931849-127.0.0.1:36145_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 191032 INFO  (zkCallback-506-thread-1) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39833_solr
   [junit4]   2> 191032 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3457a8e5{/solr,null,UNAVAILABLE}
   [junit4]   2> 191032 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 191032 INFO  (zkCallback-506-thread-1) [n:127.0.0.1:39833_solr    ] o.a.s.c.Overseer Overseer (id=72217680416931850-127.0.0.1:39833_solr-n_0000000001) starting
   [junit4]   2> 191033 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
   [junit4]   2> 191033 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 191033 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 191034 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 191034 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@282260ca{/solr,null,AVAILABLE}
   [junit4]   2> 191034 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@a7d3366{HTTP/1.1,[http/1.1]}{127.0.0.1:46653}
   [junit4]   2> 191034 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.Server Started @191070ms
   [junit4]   2> 191034 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38035}
   [junit4]   2> 191035 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 191035 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 191035 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.6.0
   [junit4]   2> 191035 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 191035 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 191035 INFO  (OverseerStateUpdate-72217680416931850-127.0.0.1:39833_solr-n_0000000001) [n:127.0.0.1:39833_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:39833_solr
   [junit4]   2> 191035 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T13:57:27.060276Z
   [junit4]   2> 191036 INFO  (zkConnectionManagerCallback-552-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191037 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 191140 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41657/solr
   [junit4]   2> 191146 INFO  (zkConnectionManagerCallback-556-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191153 INFO  (zkConnectionManagerCallback-558-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191157 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 191159 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 191159 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38035_solr
   [junit4]   2> 191160 INFO  (zkCallback-540-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191160 INFO  (zkCallback-549-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191160 INFO  (zkCallback-506-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191160 INFO  (zkCallback-493-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191160 INFO  (zkCallback-535-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191160 INFO  (zkCallback-528-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191160 INFO  (zkCallback-524-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191171 INFO  (zkCallback-557-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191179 INFO  (zkConnectionManagerCallback-565-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191180 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 191180 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41657/solr ready
   [junit4]   2> 191180 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 191189 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38035.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 191195 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38035.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 191195 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38035.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 191196 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-001/tempDir-001/node2/.
   [junit4]   2> 191221 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:46653/solr. Proxy: http://127.0.0.1:38035/solr
   [junit4]   2> 191222 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@27bcfccc{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 191222 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=84519797
   [junit4]   2> 191223 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 191223 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@40bbc784: rootName = solr_39833, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@18b81eeb
   [junit4]   2> 191229 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 191229 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@214f0ec4: rootName = solr_39833, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2e28778b
   [junit4]   2> 191232 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 191232 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3cd3a8b8: rootName = solr_39833, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@44326a59
   [junit4]   2> 191234 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:39833_solr
   [junit4]   2> 191234 INFO  (zkCallback-557-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191234 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 191234 INFO  (zkCallback-535-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191234 INFO  (zkCallback-506-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191234 INFO  (zkCallback-528-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191234 INFO  (zkCallback-524-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191234 INFO  (zkCallback-493-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191234 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 191234 INFO  (zkCallback-549-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191234 INFO  (zkCallback-540-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191235 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.Overseer Overseer (id=72217680416931850-127.0.0.1:39833_solr-n_0000000001) closing
   [junit4]   2> 191235 INFO  (zkCallback-564-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191235 WARN  (OverseerAutoScalingTriggerThread-72217680416931850-127.0.0.1:39833_solr-n_0000000001) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 191235 INFO  (OverseerStateUpdate-72217680416931850-127.0.0.1:39833_solr-n_0000000001) [n:127.0.0.1:39833_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:39833_solr
   [junit4]   2> 191238 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100919903c1000a, likely client has closed socket
   [junit4]   2> 191239 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7348b30{/solr,null,UNAVAILABLE}
   [junit4]   2> 191239 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 191239 INFO  (zkCallback-493-thread-2) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46405_solr
   [junit4]   2> 191239 INFO  (zkCallback-493-thread-2) [n:127.0.0.1:46405_solr    ] o.a.s.c.Overseer Overseer (id=72217680416931846-127.0.0.1:46405_solr-n_0000000002) starting
   [junit4]   2> 191239 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
   [junit4]   2> 191240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 191240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 191240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 191240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6341a9bb{/solr,null,AVAILABLE}
   [junit4]   2> 191240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7ccad72e{HTTP/1.1,[http/1.1]}{127.0.0.1:41611}
   [junit4]   2> 191240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.Server Started @191276ms
   [junit4]   2> 191240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46211}
   [junit4]   2> 191240 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 191240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 191241 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.6.0
   [junit4]   2> 191241 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 191241 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 191241 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T13:57:27.266119Z
   [junit4]   2> 191241 INFO  (OverseerStateUpdate-72217680416931846-127.0.0.1:46405_solr-n_0000000002) [n:127.0.0.1:46405_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:46405_solr
   [junit4]   2> 191242 INFO  (zkConnectionManagerCallback-568-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191242 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 191372 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41657/solr
   [junit4]   2> 191373 INFO  (zkConnectionManagerCallback-572-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191375 INFO  (zkConnectionManagerCallback-574-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191378 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 191379 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 191379 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46211_solr
   [junit4]   2> 191380 INFO  (zkCallback-528-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191380 INFO  (zkCallback-564-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191380 INFO  (zkCallback-535-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191380 INFO  (zkCallback-549-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191380 INFO  (zkCallback-557-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191380 INFO  (zkCallback-573-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191380 INFO  (zkCallback-540-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191380 INFO  (zkCallback-493-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191398 INFO  (zkConnectionManagerCallback-581-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191399 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 191399 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41657/solr ready
   [junit4]   2> 191399 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 191409 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46211.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 191414 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46211.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 191414 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46211.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 191415 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-001/tempDir-001/node4/.
   [junit4]   2> 191439 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:41611/solr. Proxy: http://127.0.0.1:46211/solr
   [junit4]   2> 191440 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@3c2bbef7{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 191440 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=238273728
   [junit4]   2> 191440 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 191440 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1a75b3af: rootName = solr_46405, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@46e24ae4
   [junit4]   2> 191445 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 191445 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@51593cc3: rootName = solr_46405, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2e28778b
   [junit4]   2> 191448 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 191448 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@46917148: rootName = solr_46405, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@44326a59
   [junit4]   2> 191449 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:46405_solr
   [junit4]   2> 191449 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 191449 INFO  (zkCallback-549-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191449 INFO  (zkCallback-528-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191449 INFO  (zkCallback-540-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191450 INFO  (zkCallback-580-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191450 INFO  (zkCallback-493-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191450 INFO  (zkCallback-573-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191450 INFO  (zkCallback-564-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191450 INFO  (zkCallback-557-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191450 INFO  (zkCallback-535-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191450 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.Overseer Overseer (id=72217680416931846-127.0.0.1:46405_solr-n_0000000002) closing
   [junit4]   2> 191450 INFO  (OverseerStateUpdate-72217680416931846-127.0.0.1:46405_solr-n_0000000002) [n:127.0.0.1:46405_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:46405_solr
   [junit4]   2> 191450 WARN  (OverseerAutoScalingTriggerThread-72217680416931846-127.0.0.1:46405_solr-n_0000000002) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 191456 INFO  (zkCallback-528-thread-1) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42731_solr
   [junit4]   2> 191456 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@29df7c26{/solr,null,UNAVAILABLE}
   [junit4]   2> 191456 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 191456 INFO  (zkCallback-528-thread-1) [n:127.0.0.1:42731_solr    ] o.a.s.c.Overseer Overseer (id=72217680416931854-127.0.0.1:42731_solr-n_0000000003) starting
   [junit4]   2> 191457 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
   [junit4]   2> 191457 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 191457 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 191457 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 191457 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@57cebadf{/solr,null,AVAILABLE}
   [junit4]   2> 191458 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3a3d0dcf{HTTP/1.1,[http/1.1]}{127.0.0.1:44863}
   [junit4]   2> 191458 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.Server Started @191494ms
   [junit4]   2> 191458 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39053}
   [junit4]   2> 191458 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 191458 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 191458 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.6.0
   [junit4]   2> 191458 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 191458 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 191458 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T13:57:27.483665Z
   [junit4]   2> 191462 INFO  (OverseerStateUpdate-72217680416931854-127.0.0.1:42731_solr-n_0000000003) [n:127.0.0.1:42731_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42731_solr
   [junit4]   2> 191463 INFO  (zkConnectionManagerCallback-584-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191463 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 191533 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41657/solr
   [junit4]   2> 191534 INFO  (zkConnectionManagerCallback-588-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191536 INFO  (zkConnectionManagerCallback-590-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191538 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 191540 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 191540 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39053_solr
   [junit4]   2> 191541 INFO  (zkCallback-528-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191541 INFO  (zkCallback-557-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191541 INFO  (zkCallback-573-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191541 INFO  (zkCallback-564-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191541 INFO  (zkCallback-549-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191541 INFO  (zkCallback-580-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191541 INFO  (zkCallback-589-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191541 INFO  (zkCallback-540-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191552 INFO  (zkConnectionManagerCallback-597-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191552 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 191553 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41657/solr ready
   [junit4]   2> 191553 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 191562 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39053.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 191568 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39053.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 191568 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39053.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 191568 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-001/tempDir-001/node1/.
   [junit4]   2> 191591 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:44863/solr. Proxy: http://127.0.0.1:39053/solr
   [junit4]   2> 191592 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@4998eabd{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 191592 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1135295768
   [junit4]   2> 191592 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 191592 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@199ee4b5: rootName = solr_42731, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@584fe1f1
   [junit4]   2> 191596 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 191596 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6331d9e7: rootName = solr_42731, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2e28778b
   [junit4]   2> 191599 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 191599 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1123a0df: rootName = solr_42731, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@44326a59
   [junit4]   2> 191600 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:42731_solr
   [junit4]   2> 191600 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 191600 INFO  (zkCallback-528-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191600 INFO  (zkCallback-540-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191600 INFO  (zkCallback-564-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191600 INFO  (zkCallback-557-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191601 INFO  (zkCallback-549-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191601 INFO  (zkCallback-580-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes f

[...truncated too long message...]

atYouKill.java:333)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
   [junit4]   2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
   [junit4]   2> at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue/qn-
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:786)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:398)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:398)
   [junit4]   2> at org.apache.solr.cloud.ZkDistributedQueue.offer(ZkDistributedQueue.java:321)
   [junit4]   2> at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1605)
   [junit4]   2> at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1573)
   [junit4]   2> at org.apache.solr.core.CoreContainer.unload(CoreContainer.java:1519)
   [junit4]   2> ... 40 more
   [junit4]   2>
   [junit4]   2> 1263915 WARN  (zkCallback-271-thread-4) [    ] o.a.s.c.ZkController Unable to register nodeLost path for 127.0.0.1:44457_solr
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling/nodeLost/127.0.0.1:44457_solr
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:786) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:398) ~[java/:?]
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60) ~[java/:?]
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:398) ~[java/:?]
   [junit4]   2> at org.apache.solr.cloud.ZkController.lambda$registerLiveNodesListener$3(ZkController.java:944) [java/:?]
   [junit4]   2> at org.apache.solr.common.cloud.ZkStateReader.lambda$refreshLiveNodes$3(ZkStateReader.java:793) [java/:?]
   [junit4]   2> at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4724) [?:?]
   [junit4]   2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:792) [java/:?]
   [junit4]   2> at org.apache.solr.common.cloud.ZkStateReader.access$1400(ZkStateReader.java:73) [java/:?]
   [junit4]   2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1298) [java/:?]
   [junit4]   2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1293) [java/:?]
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$1(SolrZkClient.java:270) [java/:?]
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514) [?:?]
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) [java/:?]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) [?:?]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) [?:?]
   [junit4]   2> at java.lang.Thread.run(Thread.java:844) [?:?]
   [junit4]   2> 1263916 INFO  (jetty-closer-143-thread-4) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@25519847{/solr,null,UNAVAILABLE}
   [junit4]   2> 1263917 INFO  (jetty-closer-143-thread-4) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1263917 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1263917 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33793 33793
   [junit4]   2> 1265466 INFO  (Thread-100) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33793 33793
   [junit4]   2> 1265467 WARN  (Thread-100) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 18 /solr/aliases.json
   [junit4]   2> 17 /solr/clusterprops.json
   [junit4]   2> 9 /solr/security.json
   [junit4]   2> 9 /solr/configs/conf
   [junit4]   2> 8 /solr/collections/pull_replica_error_handling_test_pull_replica_disconnects_from_zoo_keeper/terms/shard1
   [junit4]   2> 8 /solr/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/terms/shard2
   [junit4]   2> 7 /solr/collections/pull_replica_error_handling_test_cant_connect_to_leader/terms/shard1
   [junit4]   2> 6 /solr/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/terms/shard1
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 18 /solr/clusterstate.json
   [junit4]   2> 10 /solr/clusterprops.json
   [junit4]   2> 7 /solr/collections/pull_replica_error_handling_test_pull_replica_disconnects_from_zoo_keeper/state.json
   [junit4]   2> 4 /solr/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/state.json
   [junit4]   2> 4 /solr/overseer_elect/election/72217979862056968-127.0.0.1:45331_solr-n_0000000000
   [junit4]   2> 3 /solr/overseer_elect/election/72217979862056985-127.0.0.1:44959_solr-n_0000000005
   [junit4]   2> 2 /solr/collections/pull_replica_error_handling_test_cant_connect_to_leader/state.json
   [junit4]   2> 2 /solr/autoscaling.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 18 /solr/collections
   [junit4]   2> 17 /solr/live_nodes
   [junit4]   2> 2 /solr/overseer/queue
   [junit4]   2> 2 /solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 2 /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 2 /solr/overseer/collection-queue-work
   [junit4]   2> 2 /solr/overseer/queue-work
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-002
   [junit4]   2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=false): {}, locale=sr-Latn-BA, timezone=Indian/Reunion
   [junit4]   2> NOTE: Linux 4.15.0-32-generic amd64/Oracle Corporation 9.0.4 (64-bit)/cpus=8,threads=1,free=325572624,total=522715136
   [junit4]   2> NOTE: All tests run in this JVM: [TestPullReplicaErrorHandling, TestPullReplicaErrorHandling]
   [junit4] Completed [5/5 (5!)] on J2 in 638.18s, 3 tests, 1 error <<< FAILURES!

[...truncated 15 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/common-build.xml:1567: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/common-build.xml:1092: There were test failures: 5 suites, 15 tests, 5 errors [seed: 82E4B50E98F6F15C]

Total time: 21 minutes 8 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   5/5 failed: org.apache.solr.cloud.TestPullReplicaErrorHandling
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=f76a424aa2b1a29eda229e0e7b292551d96e9d29, workspace=/var/lib/jenkins/workspace/Lucene-Solr-7.x-Linux
[WARNINGS] Computing warning deltas based on reference build #2736
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk-9.0.4) - Build # 2737 - Still Unstable!

Tomás Fernández Löbbe
This is because of https://issues.apache.org/jira/browse/SOLR-12766. Just pushed a fix.

On Thu, Sep 13, 2018 at 8:32 AM Policeman Jenkins Server <[hidden email]> wrote:
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/2737/
Java: 64bit/jdk-9.0.4 -XX:-UseCompressedOops -XX:+UseParallelGC

4 tests failed.
FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:37573/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37573/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([82E4B50E98F6F15C:CE8D57EEDEE7576]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:33393/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33393/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([82E4B50E98F6F15C:CE8D57EEDEE7576]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:44959/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:44959/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([82E4B50E98F6F15C:CE8D57EEDEE7576]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:38035/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38035/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([82E4B50E98F6F15C:CE8D57EEDEE7576]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at jdk.internal.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 13356 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestPullReplicaErrorHandling
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-001/init-core-data-001
   [junit4]   2> 190633 WARN  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=69 numCloses=69
   [junit4]   2> 190633 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 190634 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl="https://issues.apache.org/jira/browse/SOLR-5776")
   [junit4]   2> 190634 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 190634 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-001/tempDir-001
   [junit4]   2> 190634 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 190634 INFO  (Thread-791) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 190634 INFO  (Thread-791) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 190639 ERROR (Thread-791) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 190734 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:41657
   [junit4]   2> 190736 INFO  (zkConnectionManagerCallback-479-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190738 INFO  (jetty-launcher-476-thread-4) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
   [junit4]   2> 190739 INFO  (jetty-launcher-476-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
   [junit4]   2> 190739 INFO  (jetty-launcher-476-thread-3) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
   [junit4]   2> 190739 INFO  (jetty-launcher-476-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
   [junit4]   2> 190744 INFO  (jetty-launcher-476-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 190744 INFO  (jetty-launcher-476-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-4) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@29df7c26{/solr,null,AVAILABLE}
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-3) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 190745 INFO  (jetty-launcher-476-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7348b30{/solr,null,AVAILABLE}
   [junit4]   2> 190746 INFO  (jetty-launcher-476-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3457a8e5{/solr,null,AVAILABLE}
   [junit4]   2> 190746 INFO  (jetty-launcher-476-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@715f57a1{/solr,null,AVAILABLE}
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1d89ee02{HTTP/1.1,[http/1.1]}{127.0.0.1:36145}
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@27bcfccc{HTTP/1.1,[http/1.1]}{127.0.0.1:39833}
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-2) [    ] o.e.j.s.Server Started @190784ms
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3c2bbef7{HTTP/1.1,[http/1.1]}{127.0.0.1:46405}
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-4) [    ] o.e.j.s.Server Started @190784ms
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4998eabd{HTTP/1.1,[http/1.1]}{127.0.0.1:42731}
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-1) [    ] o.e.j.s.Server Started @190784ms
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-3) [    ] o.e.j.s.Server Started @190784ms
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36145}
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39833}
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46405}
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42731}
   [junit4]   2> 190748 ERROR (jetty-launcher-476-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 190748 ERROR (jetty-launcher-476-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 190748 ERROR (jetty-launcher-476-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 190748 ERROR (jetty-launcher-476-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.6.0
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.6.0
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.6.0
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.6.0
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T13:57:26.773903Z
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T13:57:26.773919Z
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T13:57:26.773931Z
   [junit4]   2> 190748 INFO  (jetty-launcher-476-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T13:57:26.773904Z
   [junit4]   2> 190756 INFO  (zkConnectionManagerCallback-486-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190756 INFO  (jetty-launcher-476-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 190756 INFO  (zkConnectionManagerCallback-484-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190756 INFO  (zkConnectionManagerCallback-485-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190756 INFO  (jetty-launcher-476-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 190756 INFO  (zkConnectionManagerCallback-487-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190756 INFO  (jetty-launcher-476-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 190757 INFO  (jetty-launcher-476-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 190761 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100919903c10002, likely client has closed socket
   [junit4]   2> 190761 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100919903c10003, likely client has closed socket
   [junit4]   2> 190809 INFO  (jetty-launcher-476-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41657/solr
   [junit4]   2> 190826 INFO  (zkConnectionManagerCallback-491-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190839 INFO  (jetty-launcher-476-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41657/solr
   [junit4]   2> 190844 INFO  (zkConnectionManagerCallback-494-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190857 INFO  (zkConnectionManagerCallback-497-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190859 INFO  (jetty-launcher-476-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41657/solr
   [junit4]   2> 190863 INFO  (zkConnectionManagerCallback-505-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190864 INFO  (zkConnectionManagerCallback-502-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190870 INFO  (zkConnectionManagerCallback-507-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190870 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36145_solr
   [junit4]   2> 190871 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.c.Overseer Overseer (id=72217680416931849-127.0.0.1:36145_solr-n_0000000000) starting
   [junit4]   2> 190885 INFO  (jetty-launcher-476-thread-4) [n:127.0.0.1:39833_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39833_solr
   [junit4]   2> 190887 INFO  (zkCallback-501-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 190888 INFO  (zkCallback-506-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 190889 INFO  (zkConnectionManagerCallback-516-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190890 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 190891 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41657/solr ready
   [junit4]   2> 190892 INFO  (jetty-launcher-476-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41657/solr
   [junit4]   2> 190894 INFO  (OverseerStateUpdate-72217680416931849-127.0.0.1:36145_solr-n_0000000000) [n:127.0.0.1:36145_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:36145_solr
   [junit4]   2> 190898 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 190898 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36145_solr
   [junit4]   2> 190904 INFO  (zkConnectionManagerCallback-520-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190905 INFO  (zkCallback-515-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 190907 INFO  (zkCallback-506-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 190907 INFO  (zkCallback-501-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 190909 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 190910 INFO  (zkConnectionManagerCallback-525-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190910 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 190910 INFO  (jetty-launcher-476-thread-4) [n:127.0.0.1:39833_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 190911 INFO  (jetty-launcher-476-thread-4) [n:127.0.0.1:39833_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41657/solr ready
   [junit4]   2> 190911 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 190911 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46405_solr
   [junit4]   2> 190912 INFO  (zkCallback-506-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 190918 INFO  (zkConnectionManagerCallback-529-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190922 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 190922 INFO  (jetty-launcher-476-thread-4) [n:127.0.0.1:39833_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 190923 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36145.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190924 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 190924 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42731_solr
   [junit4]   2> 190925 INFO  (zkCallback-506-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 190928 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36145.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190928 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36145.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190929 INFO  (jetty-launcher-476-thread-2) [n:127.0.0.1:36145_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-001/tempDir-001/node2/.
   [junit4]   2> 190929 INFO  (zkCallback-493-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 190930 INFO  (zkCallback-493-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 190931 INFO  (zkCallback-515-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
   [junit4]   2> 190931 INFO  (zkCallback-524-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
   [junit4]   2> 190931 INFO  (zkCallback-501-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
   [junit4]   2> 190931 INFO  (zkCallback-528-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 190933 INFO  (zkConnectionManagerCallback-536-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190934 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 190938 INFO  (jetty-launcher-476-thread-4) [n:127.0.0.1:39833_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39833.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190939 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41657/solr ready
   [junit4]   2> 190944 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 190944 INFO  (jetty-launcher-476-thread-4) [n:127.0.0.1:39833_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39833.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190944 INFO  (zkConnectionManagerCallback-541-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 190944 INFO  (jetty-launcher-476-thread-4) [n:127.0.0.1:39833_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39833.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190945 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 190945 INFO  (jetty-launcher-476-thread-4) [n:127.0.0.1:39833_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-001/tempDir-001/node4/.
   [junit4]   2> 190945 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41657/solr ready
   [junit4]   2> 190946 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 190953 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46405.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190955 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42731.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190958 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46405.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190958 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46405.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190959 INFO  (jetty-launcher-476-thread-1) [n:127.0.0.1:46405_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-001/tempDir-001/node1/.
   [junit4]   2> 190959 WARN  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.m.r.j.JmxMetricsReporter Unable to register gauge
   [junit4]   2> javax.management.InstanceNotFoundException: solr_42731:dom1=solr,dom2=jvm,name0=memory,name1=pools,name2=CodeHeap-'profiled-nmethods',name=committed
   [junit4]   2>        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1083) ~[?:?]
   [junit4]   2>        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:423) ~[?:?]
   [junit4]   2>        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:411) ~[?:?]
   [junit4]   2>        at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546) ~[?:?]
   [junit4]   2>        at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.registerMBean(JmxMetricsReporter.java:530) ~[java/:?]
   [junit4]   2>        at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.onGaugeAdded(JmxMetricsReporter.java:565) [java/:?]
   [junit4]   2>        at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter.lambda$start$0(JmxMetricsReporter.java:738) [java/:?]
   [junit4]   2>        at java.util.HashMap.forEach(HashMap.java:1341) [?:?]
   [junit4]   2>        at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter.start(JmxMetricsReporter.java:728) [java/:?]
   [junit4]   2>        at org.apache.solr.metrics.reporters.SolrJmxReporter.doInit(SolrJmxReporter.java:110) [java/:?]
   [junit4]   2>        at org.apache.solr.metrics.SolrMetricReporter.init(SolrMetricReporter.java:70) [java/:?]
   [junit4]   2>        at org.apache.solr.metrics.SolrMetricManager.loadReporter(SolrMetricManager.java:941) [java/:?]
   [junit4]   2>        at org.apache.solr.metrics.SolrMetricManager.loadReporters(SolrMetricManager.java:868) [java/:?]
   [junit4]   2>        at org.apache.solr.core.CoreContainer.load(CoreContainer.java:590) [java/:?]
   [junit4]   2>        at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:252) [java/:?]
   [junit4]   2>        at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:172) [java/:?]
   [junit4]   2>        at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2>        at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2>        at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2>        at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2>        at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2>        at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2>        at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2>        at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) [java/:?]
   [junit4]   2>        at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2>        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) [jetty-util-9.4.11.v20180605.jar:9.4.11.v20180605]
   [junit4]   2>        at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) [java/:?]
   [junit4]   2>        at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) [java/:?]
   [junit4]   2>        at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) [java/:?]
   [junit4]   2>        at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) [java/:?]
   [junit4]   2>        at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
   [junit4]   2>        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) [java/:?]
   [junit4]   2>        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) [?:?]
   [junit4]   2>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) [?:?]
   [junit4]   2>        at java.lang.Thread.run(Thread.java:844) [?:?]
   [junit4]   2> 190968 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42731.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190968 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42731.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 190969 INFO  (jetty-launcher-476-thread-3) [n:127.0.0.1:42731_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-001/tempDir-001/node3/.
   [junit4]   2> 191012 INFO  (zkConnectionManagerCallback-545-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191014 INFO  (zkConnectionManagerCallback-550-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191015 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 191015 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41657/solr ready
   [junit4]   2> 191018 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@1d89ee02{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 191018 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1843628192
   [junit4]   2> 191019 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 191019 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5d0f8bbd: rootName = solr_36145, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@3304304
   [junit4]   2> 191023 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 191023 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3dfe3db1: rootName = solr_36145, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2e28778b
   [junit4]   2> 191025 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 191025 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5dce3633: rootName = solr_36145, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@44326a59
   [junit4]   2> 191026 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:36145_solr
   [junit4]   2> 191026 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 191026 INFO  (zkCallback-515-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191026 INFO  (zkCallback-528-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191026 INFO  (zkCallback-524-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191027 INFO  (zkCallback-506-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191026 INFO  (zkCallback-501-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191027 INFO  (zkCallback-493-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191027 INFO  (zkCallback-535-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191028 INFO  (zkCallback-540-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191029 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.Overseer Overseer (id=72217680416931849-127.0.0.1:36145_solr-n_0000000000) closing
   [junit4]   2> 191029 INFO  (zkCallback-549-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191029 INFO  (OverseerStateUpdate-72217680416931849-127.0.0.1:36145_solr-n_0000000000) [n:127.0.0.1:36145_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36145_solr
   [junit4]   2> 191029 WARN  (OverseerAutoScalingTriggerThread-72217680416931849-127.0.0.1:36145_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 191032 INFO  (zkCallback-506-thread-1) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39833_solr
   [junit4]   2> 191032 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3457a8e5{/solr,null,UNAVAILABLE}
   [junit4]   2> 191032 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 191032 INFO  (zkCallback-506-thread-1) [n:127.0.0.1:39833_solr    ] o.a.s.c.Overseer Overseer (id=72217680416931850-127.0.0.1:39833_solr-n_0000000001) starting
   [junit4]   2> 191033 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
   [junit4]   2> 191033 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 191033 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 191034 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 191034 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@282260ca{/solr,null,AVAILABLE}
   [junit4]   2> 191034 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@a7d3366{HTTP/1.1,[http/1.1]}{127.0.0.1:46653}
   [junit4]   2> 191034 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.Server Started @191070ms
   [junit4]   2> 191034 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38035}
   [junit4]   2> 191035 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 191035 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 191035 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.6.0
   [junit4]   2> 191035 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 191035 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 191035 INFO  (OverseerStateUpdate-72217680416931850-127.0.0.1:39833_solr-n_0000000001) [n:127.0.0.1:39833_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:39833_solr
   [junit4]   2> 191035 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T13:57:27.060276Z
   [junit4]   2> 191036 INFO  (zkConnectionManagerCallback-552-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191037 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 191140 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41657/solr
   [junit4]   2> 191146 INFO  (zkConnectionManagerCallback-556-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191153 INFO  (zkConnectionManagerCallback-558-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191157 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 191159 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 191159 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38035_solr
   [junit4]   2> 191160 INFO  (zkCallback-540-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191160 INFO  (zkCallback-549-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191160 INFO  (zkCallback-506-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191160 INFO  (zkCallback-493-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191160 INFO  (zkCallback-535-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191160 INFO  (zkCallback-528-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191160 INFO  (zkCallback-524-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191171 INFO  (zkCallback-557-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 191179 INFO  (zkConnectionManagerCallback-565-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191180 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 191180 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41657/solr ready
   [junit4]   2> 191180 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 191189 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38035.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 191195 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38035.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 191195 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38035.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@69625478
   [junit4]   2> 191196 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [n:127.0.0.1:38035_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_82E4B50E98F6F15C-001/tempDir-001/node2/.
   [junit4]   2> 191221 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:46653/solr. Proxy: http://127.0.0.1:38035/solr
   [junit4]   2> 191222 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@27bcfccc{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 191222 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=84519797
   [junit4]   2> 191223 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 191223 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@40bbc784: rootName = solr_39833, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@18b81eeb
   [junit4]   2> 191229 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 191229 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@214f0ec4: rootName = solr_39833, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2e28778b
   [junit4]   2> 191232 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 191232 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3cd3a8b8: rootName = solr_39833, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@44326a59
   [junit4]   2> 191234 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:39833_solr
   [junit4]   2> 191234 INFO  (zkCallback-557-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191234 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 191234 INFO  (zkCallback-535-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191234 INFO  (zkCallback-506-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191234 INFO  (zkCallback-528-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191234 INFO  (zkCallback-524-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191234 INFO  (zkCallback-493-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191234 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 191234 INFO  (zkCallback-549-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191234 INFO  (zkCallback-540-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191235 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.Overseer Overseer (id=72217680416931850-127.0.0.1:39833_solr-n_0000000001) closing
   [junit4]   2> 191235 INFO  (zkCallback-564-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 191235 WARN  (OverseerAutoScalingTriggerThread-72217680416931850-127.0.0.1:39833_solr-n_0000000001) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 191235 INFO  (OverseerStateUpdate-72217680416931850-127.0.0.1:39833_solr-n_0000000001) [n:127.0.0.1:39833_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:39833_solr
   [junit4]   2> 191238 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100919903c1000a, likely client has closed socket
   [junit4]   2> 191239 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7348b30{/solr,null,UNAVAILABLE}
   [junit4]   2> 191239 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 191239 INFO  (zkCallback-493-thread-2) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46405_solr
   [junit4]   2> 191239 INFO  (zkCallback-493-thread-2) [n:127.0.0.1:46405_solr    ] o.a.s.c.Overseer Overseer (id=72217680416931846-127.0.0.1:46405_solr-n_0000000002) starting
   [junit4]   2> 191239 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11
   [junit4]   2> 191240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 191240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 191240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 191240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6341a9bb{/solr,null,AVAILABLE}
   [junit4]   2> 191240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7ccad72e{HTTP/1.1,[http/1.1]}{127.0.0.1:41611}
   [junit4]   2> 191240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.e.j.s.Server Started @191276ms
   [junit4]   2> 191240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46211}
   [junit4]   2> 191240 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 191240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 191241 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.6.0
   [junit4]   2> 191241 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 191241 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 191241 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[82E4B50E98F6F15C]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T13:57:27.266119Z
   [junit4]   2> 191241 INFO  (OverseerStateUpdate-72217680416931846-127.0.0.1:46405_solr-n_0000000002) [n:127.0.0.1:46405_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:46405_solr
   [junit4]   2> 191242 INFO  (zkConnectionManagerCallback-568-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 191242 INFO  (SUITE-
Reply | Threaded
Open this post in threaded view
|

Re: [JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk-10.0.1) - Build # 2736 - Still Unstable!

Tomás Fernández Löbbe
In reply to this post by Policeman Jenkins Server-2
This is because of https://issues.apache.org/jira/browse/SOLR-12766. Just pushed a fix.

On Thu, Sep 13, 2018 at 3:53 AM Policeman Jenkins Server <[hidden email]> wrote:
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/2736/
Java: 64bit/jdk-10.0.1 -XX:-UseCompressedOops -XX:+UseSerialGC

4 tests failed.
FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:38355/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38355/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([C22EB685876604C5:4C22D6F5F27E80EF]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:42281/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42281/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([C22EB685876604C5:4C22D6F5F27E80EF]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:40459/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:40459/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([C22EB685876604C5:4C22D6F5F27E80EF]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:39463/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39463/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([C22EB685876604C5:4C22D6F5F27E80EF]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at jdk.internal.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 12984 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestPullReplicaErrorHandling
   [junit4]   2> 28912 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/init-core-data-001
   [junit4]   2> 28913 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 28914 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl="https://issues.apache.org/jira/browse/SOLR-5776")
   [junit4]   2> 28914 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/tempDir-001
   [junit4]   2> 28914 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 28915 INFO  (Thread-53) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 28915 INFO  (Thread-53) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 28925 ERROR (Thread-53) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 29015 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:37551
   [junit4]   2> 29041 INFO  (zkConnectionManagerCallback-77-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29064 INFO  (jetty-launcher-74-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10.0.1+10
   [junit4]   2> 29082 INFO  (jetty-launcher-74-thread-3) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10.0.1+10
   [junit4]   2> 29082 INFO  (jetty-launcher-74-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10.0.1+10
   [junit4]   2> 29083 INFO  (jetty-launcher-74-thread-4) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10.0.1+10
   [junit4]   2> 29249 INFO  (jetty-launcher-74-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 29249 INFO  (jetty-launcher-74-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 29250 INFO  (jetty-launcher-74-thread-2) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 29250 INFO  (jetty-launcher-74-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2b17399{/solr,null,AVAILABLE}
   [junit4]   2> 29255 INFO  (jetty-launcher-74-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@45efd000{HTTP/1.1,[http/1.1]}{127.0.0.1:44983}
   [junit4]   2> 29255 INFO  (jetty-launcher-74-thread-2) [    ] o.e.j.s.Server Started @29280ms
   [junit4]   2> 29255 INFO  (jetty-launcher-74-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44983}
   [junit4]   2> 29255 ERROR (jetty-launcher-74-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 29255 INFO  (jetty-launcher-74-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 29256 INFO  (jetty-launcher-74-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.6.0
   [junit4]   2> 29256 INFO  (jetty-launcher-74-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 29256 INFO  (jetty-launcher-74-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 29256 INFO  (jetty-launcher-74-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T09:11:05.990176Z
   [junit4]   2> 29259 INFO  (jetty-launcher-74-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 29259 INFO  (jetty-launcher-74-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 29259 INFO  (jetty-launcher-74-thread-3) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 29260 INFO  (jetty-launcher-74-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1dcd20f2{/solr,null,AVAILABLE}
   [junit4]   2> 29261 INFO  (jetty-launcher-74-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@76e76d63{HTTP/1.1,[http/1.1]}{127.0.0.1:35337}
   [junit4]   2> 29261 INFO  (jetty-launcher-74-thread-3) [    ] o.e.j.s.Server Started @29286ms
   [junit4]   2> 29261 INFO  (jetty-launcher-74-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35337}
   [junit4]   2> 29262 ERROR (jetty-launcher-74-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 29262 INFO  (jetty-launcher-74-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 29262 INFO  (jetty-launcher-74-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.6.0
   [junit4]   2> 29262 INFO  (jetty-launcher-74-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 29262 INFO  (jetty-launcher-74-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 29262 INFO  (jetty-launcher-74-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T09:11:05.996701Z
   [junit4]   2> 29273 INFO  (jetty-launcher-74-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 29274 INFO  (jetty-launcher-74-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 29274 INFO  (jetty-launcher-74-thread-4) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 29291 INFO  (jetty-launcher-74-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@43b815cd{/solr,null,AVAILABLE}
   [junit4]   2> 29292 INFO  (jetty-launcher-74-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2780c7de{HTTP/1.1,[http/1.1]}{127.0.0.1:43995}
   [junit4]   2> 29292 INFO  (jetty-launcher-74-thread-4) [    ] o.e.j.s.Server Started @29317ms
   [junit4]   2> 29292 INFO  (jetty-launcher-74-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43995}
   [junit4]   2> 29292 ERROR (jetty-launcher-74-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 29292 INFO  (jetty-launcher-74-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 29292 INFO  (jetty-launcher-74-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.6.0
   [junit4]   2> 29292 INFO  (jetty-launcher-74-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 29293 INFO  (jetty-launcher-74-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 29293 INFO  (jetty-launcher-74-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T09:11:06.027030Z
   [junit4]   2> 29293 INFO  (jetty-launcher-74-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 29293 INFO  (jetty-launcher-74-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 29293 INFO  (jetty-launcher-74-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 29302 INFO  (zkConnectionManagerCallback-81-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29306 INFO  (jetty-launcher-74-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1c7e60e5{/solr,null,AVAILABLE}
   [junit4]   2> 29306 INFO  (jetty-launcher-74-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 29307 INFO  (jetty-launcher-74-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5b8004eb{HTTP/1.1,[http/1.1]}{127.0.0.1:34527}
   [junit4]   2> 29307 INFO  (jetty-launcher-74-thread-1) [    ] o.e.j.s.Server Started @29332ms
   [junit4]   2> 29307 INFO  (jetty-launcher-74-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34527}
   [junit4]   2> 29307 ERROR (jetty-launcher-74-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 29307 INFO  (jetty-launcher-74-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 29307 INFO  (jetty-launcher-74-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.6.0
   [junit4]   2> 29307 INFO  (jetty-launcher-74-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 29307 INFO  (jetty-launcher-74-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 29307 INFO  (jetty-launcher-74-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T09:11:06.041726Z
   [junit4]   2> 29315 INFO  (zkConnectionManagerCallback-83-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29329 INFO  (jetty-launcher-74-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 29343 INFO  (zkConnectionManagerCallback-79-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29344 INFO  (jetty-launcher-74-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 29377 INFO  (zkConnectionManagerCallback-85-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29382 INFO  (jetty-launcher-74-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 29554 INFO  (jetty-launcher-74-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37551/solr
   [junit4]   2> 29598 INFO  (jetty-launcher-74-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37551/solr
   [junit4]   2> 29618 INFO  (zkConnectionManagerCallback-90-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29626 INFO  (zkConnectionManagerCallback-93-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29637 INFO  (zkConnectionManagerCallback-95-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29649 INFO  (zkConnectionManagerCallback-97-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29694 INFO  (jetty-launcher-74-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37551/solr
   [junit4]   2> 29777 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43995_solr
   [junit4]   2> 29777 INFO  (zkConnectionManagerCallback-105-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29779 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.c.Overseer Overseer (id=72216554442522631-127.0.0.1:43995_solr-n_0000000000) starting
   [junit4]   2> 29798 INFO  (jetty-launcher-74-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37551/solr
   [junit4]   2> 29842 INFO  (jetty-launcher-74-thread-3) [n:127.0.0.1:35337_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35337_solr
   [junit4]   2> 29842 INFO  (zkConnectionManagerCallback-109-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29917 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 29921 INFO  (zkCallback-96-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 29933 INFO  (zkConnectionManagerCallback-111-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 29953 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 29957 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 29957 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34527_solr
   [junit4]   2> 29970 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 29973 INFO  (zkCallback-96-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 30030 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 30033 INFO  (zkConnectionManagerCallback-119-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30033 INFO  (zkConnectionManagerCallback-114-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30049 INFO  (zkConnectionManagerCallback-123-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30058 INFO  (jetty-launcher-74-thread-3) [n:127.0.0.1:35337_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 30060 INFO  (jetty-launcher-74-thread-3) [n:127.0.0.1:35337_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37551/solr ready
   [junit4]   2> 30064 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 30065 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37551/solr ready
   [junit4]   2> 30066 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 30073 INFO  (zkConnectionManagerCallback-130-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30079 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 30079 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44983_solr
   [junit4]   2> 30084 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 30084 INFO  (zkCallback-96-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 30084 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 30085 INFO  (OverseerStateUpdate-72216554442522631-127.0.0.1:43995_solr-n_0000000000) [n:127.0.0.1:43995_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:43995_solr
   [junit4]   2> 30093 INFO  (jetty-launcher-74-thread-3) [n:127.0.0.1:35337_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 30103 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 30103 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43995_solr
   [junit4]   2> 30109 INFO  (zkCallback-118-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 30122 INFO  (jetty-launcher-74-thread-3) [n:127.0.0.1:35337_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35337.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30150 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 30150 INFO  (zkCallback-96-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 30150 INFO  (zkCallback-118-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 30151 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 30151 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 30152 INFO  (jetty-launcher-74-thread-3) [n:127.0.0.1:35337_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35337.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30152 INFO  (jetty-launcher-74-thread-3) [n:127.0.0.1:35337_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35337.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30153 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37551/solr ready
   [junit4]   2> 30153 INFO  (jetty-launcher-74-thread-3) [n:127.0.0.1:35337_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/tempDir-001/node3/.
   [junit4]   2> 30173 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 30214 INFO  (zkCallback-113-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
   [junit4]   2> 30214 INFO  (zkCallback-122-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 30233 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43995.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30241 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43995.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30241 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43995.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30245 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 30249 INFO  (jetty-launcher-74-thread-4) [n:127.0.0.1:43995_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/tempDir-001/node4/.
   [junit4]   2> 30259 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34527.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30266 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34527.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30266 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34527.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30268 INFO  (jetty-launcher-74-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/tempDir-001/node1/.
   [junit4]   2> 30305 INFO  (zkConnectionManagerCallback-138-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30314 INFO  (zkCallback-122-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 30369 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 30373 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37551/solr ready
   [junit4]   2> 30417 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 30434 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44983.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30441 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44983.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30442 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44983.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 30443 INFO  (jetty-launcher-74-thread-2) [n:127.0.0.1:44983_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/tempDir-001/node2/.
   [junit4]   2> 30575 INFO  (zkConnectionManagerCallback-143-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30645 INFO  (zkConnectionManagerCallback-148-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30647 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 30647 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37551/solr ready
   [junit4]   2> 30654 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@76e76d63{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 30662 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=665933534
   [junit4]   2> 30662 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 30662 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@62778794: rootName = solr_35337, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@1ebc0f06
   [junit4]   2> 30670 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 30670 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@74bc4a0c: rootName = solr_35337, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@66ed5911
   [junit4]   2> 30675 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 30675 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@75b24664: rootName = solr_35337, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6b576573
   [junit4]   2> 30680 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:35337_solr
   [junit4]   2> 30681 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30681 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30681 INFO  (zkCallback-122-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30681 INFO  (zkCallback-118-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30681 INFO  (zkCallback-113-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30682 INFO  (zkCallback-96-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30683 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 30698 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 30731 INFO  (zkCallback-147-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30731 INFO  (zkCallback-129-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30735 INFO  (zkCallback-137-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 30735 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1dcd20f2{/solr,null,UNAVAILABLE}
   [junit4]   2> 30735 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 30736 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10.0.1+10
   [junit4]   2> 30864 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 30864 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 30864 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 30870 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@215ebd2f{/solr,null,AVAILABLE}
   [junit4]   2> 30886 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@f8a8afc{HTTP/1.1,[http/1.1]}{127.0.0.1:33583}
   [junit4]   2> 30886 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.Server Started @30911ms
   [junit4]   2> 30886 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39463}
   [junit4]   2> 30887 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 30887 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 30887 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.6.0
   [junit4]   2> 30887 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 30887 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 30887 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T09:11:07.621582Z
   [junit4]   2> 30933 INFO  (zkConnectionManagerCallback-150-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 30935 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 31446 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37551/solr
   [junit4]   2> 31462 INFO  (zkConnectionManagerCallback-154-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 31478 INFO  (zkConnectionManagerCallback-156-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 31484 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 31488 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 31488 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39463_solr
   [junit4]   2> 31500 INFO  (zkCallback-113-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 31500 INFO  (zkCallback-137-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 31500 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 31500 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 31500 INFO  (zkCallback-147-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 31500 INFO  (zkCallback-129-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 31502 INFO  (zkCallback-118-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 31512 INFO  (zkCallback-155-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 31587 INFO  (zkConnectionManagerCallback-163-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 31592 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 31593 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37551/solr ready
   [junit4]   2> 31617 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 31646 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39463.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 31672 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39463.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 31673 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39463.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 31678 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [n:127.0.0.1:39463_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/tempDir-001/node3/.
   [junit4]   2> 31787 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:33583/solr. Proxy: http://127.0.0.1:39463/solr
   [junit4]   2> 31790 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@2780c7de{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 31791 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=211244492
   [junit4]   2> 31791 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 31791 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@37e3d6a7: rootName = solr_43995, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@65c4d10e
   [junit4]   2> 31799 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 31799 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@738a3b72: rootName = solr_43995, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@66ed5911
   [junit4]   2> 31803 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 31804 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6038370: rootName = solr_43995, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6b576573
   [junit4]   2> 31808 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:43995_solr
   [junit4]   2> 31809 INFO  (zkCallback-94-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31809 INFO  (zkCallback-118-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31809 INFO  (zkCallback-129-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31809 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31809 INFO  (zkCallback-137-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31809 INFO  (zkCallback-113-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31809 INFO  (zkCallback-155-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31810 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 31814 INFO  (zkCallback-147-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31818 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.Overseer Overseer (id=72216554442522631-127.0.0.1:43995_solr-n_0000000000) closing
   [junit4]   2> 31818 INFO  (OverseerStateUpdate-72216554442522631-127.0.0.1:43995_solr-n_0000000000) [n:127.0.0.1:43995_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:43995_solr
   [junit4]   2> 31838 WARN  (OverseerAutoScalingTriggerThread-72216554442522631-127.0.0.1:43995_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 31839 INFO  (zkCallback-162-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 31848 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@43b815cd{/solr,null,UNAVAILABLE}
   [junit4]   2> 31848 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 31849 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34527_solr
   [junit4]   2> 31850 INFO  (zkCallback-110-thread-1) [n:127.0.0.1:34527_solr    ] o.a.s.c.Overseer Overseer (id=72216554442522635-127.0.0.1:34527_solr-n_0000000002) starting
   [junit4]   2> 31850 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10.0.1+10
   [junit4]   2> 31925 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 31925 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 31925 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 31938 INFO  (OverseerStateUpdate-72216554442522635-127.0.0.1:34527_solr-n_0000000002) [n:127.0.0.1:34527_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:34527_solr
   [junit4]   2> 31965 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2b8b4b60{/solr,null,AVAILABLE}
   [junit4]   2> 31966 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3e1abbc{HTTP/1.1,[http/1.1]}{127.0.0.1:45319}
   [junit4]   2> 31966 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.Server Started @31991ms
   [junit4]   2> 31966 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41137}
   [junit4]   2> 31966 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 31978 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 31978 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 7.6.0
   [junit4]   2> 31978 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 31978 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 31978 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T09:11:08.712531Z
   [junit4]   2> 32069 INFO  (zkConnectionManagerCallback-166-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 32071 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 32205 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37551/solr
   [junit4]   2> 32212 INFO  (zkConnectionManagerCallback-170-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 32216 INFO  (zkConnectionManagerCallback-172-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 32220 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 32223 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 32223 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41137_solr
   [junit4]   2> 32224 INFO  (zkCallback-147-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32224 INFO  (zkCallback-113-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32224 INFO  (zkCallback-129-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32224 INFO  (zkCallback-155-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32224 INFO  (zkCallback-137-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32224 INFO  (zkCallback-162-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32224 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32224 INFO  (zkCallback-171-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 32253 INFO  (zkConnectionManagerCallback-179-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 32255 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 32259 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37551/solr ready
   [junit4]   2> 32259 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 32271 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41137.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 32277 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41137.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 32277 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41137.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1728c79e
   [junit4]   2> 32278 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestPullReplicaErrorHandling_C22EB685876604C5-001/tempDir-001/node4/.
   [junit4]   2> 32333 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:45319/solr. Proxy: http://127.0.0.1:41137/solr
   [junit4]   2> 32334 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@5b8004eb{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 32334 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1909036678
   [junit4]   2> 32335 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 32335 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2ad30f93: rootName = solr_34527, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@2e00451d
   [junit4]   2> 32342 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 32342 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@27482a76: rootName = solr_34527, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@66ed5911
   [junit4]   2> 32345 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 32345 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1f360f5b: rootName = solr_34527, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6b576573
   [junit4]   2> 32346 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:34527_solr
   [junit4]   2> 32347 INFO  (zkCallback-155-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32347 INFO  (zkCallback-147-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32347 INFO  (zkCallback-162-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32347 INFO  (zkCallback-113-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32347 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[C22EB685876604C5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 32348 INFO  (zkCallback-110-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 32350