[JENKINS] Lucene-Solr-Tests-master - Build # 2086 - Failure

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2086 - Failure

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2086/

2 tests failed.
FAILED:  org.apache.solr.metrics.reporters.SolrSlf4jReporterTest.testReporter

Error Message:
Test requires that log-level is at-least INFO, but INFO is disabled

Stack Trace:
java.lang.AssertionError: Test requires that log-level is at-least INFO, but INFO is disabled
        at __randomizedtesting.SeedInfo.seed([465B50C8C4528230:19BF7DFFAF5E1175]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.metrics.reporters.SolrSlf4jReporterTest.ensureLoggingConfiguredAppropriately(SolrSlf4jReporterTest.java:99)
        at org.apache.solr.metrics.reporters.SolrSlf4jReporterTest.testReporter(SolrSlf4jReporterTest.java:49)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.update.processor.UpdateRequestProcessorFactoryTest.testUpdateDistribChainSkipping

Error Message:
Tests must be run with INFO level logging otherwise LogUpdateProcessor isn't used and can't be tested.

Stack Trace:
java.lang.AssertionError: Tests must be run with INFO level logging otherwise LogUpdateProcessor isn't used and can't be tested.
        at __randomizedtesting.SeedInfo.seed([465B50C8C4528230:37BFAE1EB2B9A5FC]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.update.processor.UpdateRequestProcessorFactoryTest.testUpdateDistribChainSkipping(UpdateRequestProcessorFactoryTest.java:101)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 11829 lines...]
   [junit4] Suite: org.apache.solr.metrics.reporters.SolrSlf4jReporterTest
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.metrics.reporters.SolrSlf4jReporterTest_465B50C8C4528230-001/init-core-data-001
   [junit4]   2> 1360874 WARN  (SUITE-SolrSlf4jReporterTest-seed#[465B50C8C4528230]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SolrSlf4jReporterTest -Dtests.method=testReporter -Dtests.seed=465B50C8C4528230 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=fr-FR -Dtests.timezone=Etc/GMT-8 -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 0.00s J1 | SolrSlf4jReporterTest.testReporter <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Test requires that log-level is at-least INFO, but INFO is disabled
   [junit4]    > at __randomizedtesting.SeedInfo.seed([465B50C8C4528230:19BF7DFFAF5E1175]:0)
   [junit4]    > at org.apache.solr.metrics.reporters.SolrSlf4jReporterTest.ensureLoggingConfiguredAppropriately(SolrSlf4jReporterTest.java:99)
   [junit4]    > at org.apache.solr.metrics.reporters.SolrSlf4jReporterTest.testReporter(SolrSlf4jReporterTest.java:49)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.metrics.reporters.SolrSlf4jReporterTest_465B50C8C4528230-001
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false): {}, locale=fr-FR, timezone=Etc/GMT-8
   [junit4]   2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=131699240,total=505937920
   [junit4]   2> NOTE: All tests run in this JVM: [TestSolrFieldCacheBean, TestAtomicUpdateErrorCases, CurrencyFieldTypeTest, TestMiniSolrCloudClusterSSL, TestSolrCLIRunExample, TestHashPartitioner, TestDistributedGrouping, CoreMergeIndexesAdminHandlerTest, TestDynamicFieldCollectionResource, TestBackupRepositoryFactory, TestSmileRequest, TestPerFieldSimilarity, BasicAuthIntegrationTest, SubstringBytesRefFilterTest, TestInfoStreamLogging, JSONWriterTest, BitVectorTest, TestJsonFacetRefinement, DeleteReplicaTest, DistanceFunctionTest, ResponseHeaderTest, IndexSchemaRuntimeFieldTest, TestBM25SimilarityFactory, TestCustomStream, SuggestComponentContextFilterQueryTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, TestScoreJoinQPNoScore, TestLeaderInitiatedRecoveryThread, HdfsLockFactoryTest, OpenCloseCoreStressTest, AddBlockUpdateTest, IndexSchemaTest, TestStressVersions, TestManagedSchemaThreadSafety, TestSchemaNameResource, CollectionsAPIDistributedZkTest, TestElisionMultitermQuery, ConnectionManagerTest, CloudMLTQParserTest, ForceLeaderTest, CSVRequestHandlerTest, TestComplexPhraseLeadingWildcard, SoftAutoCommitTest, FileBasedSpellCheckerTest, TestHalfAndHalfDocValues, TestCoreContainer, TestExpandComponent, TestXIncludeConfig, TolerantUpdateProcessorTest, RestartWhileUpdatingTest, TestCursorMarkWithoutUniqueKey, HdfsBasicDistributedZk2Test, AlternateDirectoryTest, DistributedExpandComponentTest, ConfigureRecoveryStrategyTest, SOLR749Test, TestFuzzyAnalyzedSuggestions, EnumFieldTest, BaseCdcrDistributedZkTest, TestConfigSetProperties, TestPartialUpdateDeduplication, TestFieldSortValues, SolrSlf4jReporterTest]
   [junit4] Completed [163/732 (1!)] on J1 in 0.16s, 1 test, 1 failure <<< FAILURES!

[...truncated 229 lines...]
   [junit4] Suite: org.apache.solr.update.processor.UpdateRequestProcessorFactoryTest
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.processor.UpdateRequestProcessorFactoryTest_465B50C8C4528230-001/init-core-data-001
   [junit4]   2> 2042588 WARN  (coreLoadExecutor-3048-thread-1) [    x:collection1] o.a.s.c.RequestHandlers no default request handler is registered (either '/select' or 'standard')
   [junit4]   2> 2042607 WARN  (coreLoadExecutor-3048-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=UpdateRequestProcessorFactoryTest -Dtests.method=testUpdateDistribChainSkipping -Dtests.seed=465B50C8C4528230 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=ar -Dtests.timezone=America/Anguilla -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 0.00s J1 | UpdateRequestProcessorFactoryTest.testUpdateDistribChainSkipping <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Tests must be run with INFO level logging otherwise LogUpdateProcessor isn't used and can't be tested.
   [junit4]    > at __randomizedtesting.SeedInfo.seed([465B50C8C4528230:37BFAE1EB2B9A5FC]:0)
   [junit4]    > at org.apache.solr.update.processor.UpdateRequestProcessorFactoryTest.testUpdateDistribChainSkipping(UpdateRequestProcessorFactoryTest.java:101)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.processor.UpdateRequestProcessorFactoryTest_465B50C8C4528230-001
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true): {}, locale=ar, timezone=America/Anguilla
   [junit4]   2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=174885808,total=512229376
   [junit4]   2> NOTE: All tests run in this JVM: [TestSolrFieldCacheBean, TestAtomicUpdateErrorCases, CurrencyFieldTypeTest, TestMiniSolrCloudClusterSSL, TestSolrCLIRunExample, TestHashPartitioner, TestDistributedGrouping, CoreMergeIndexesAdminHandlerTest, TestDynamicFieldCollectionResource, TestBackupRepositoryFactory, TestSmileRequest, TestPerFieldSimilarity, BasicAuthIntegrationTest, SubstringBytesRefFilterTest, TestInfoStreamLogging, JSONWriterTest, BitVectorTest, TestJsonFacetRefinement, DeleteReplicaTest, DistanceFunctionTest, ResponseHeaderTest, IndexSchemaRuntimeFieldTest, TestBM25SimilarityFactory, TestCustomStream, SuggestComponentContextFilterQueryTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, TestScoreJoinQPNoScore, TestLeaderInitiatedRecoveryThread, HdfsLockFactoryTest, OpenCloseCoreStressTest, AddBlockUpdateTest, IndexSchemaTest, TestStressVersions, TestManagedSchemaThreadSafety, TestSchemaNameResource, CollectionsAPIDistributedZkTest, TestElisionMultitermQuery, ConnectionManagerTest, CloudMLTQParserTest, ForceLeaderTest, CSVRequestHandlerTest, TestComplexPhraseLeadingWildcard, SoftAutoCommitTest, FileBasedSpellCheckerTest, TestHalfAndHalfDocValues, TestCoreContainer, TestExpandComponent, TestXIncludeConfig, TolerantUpdateProcessorTest, RestartWhileUpdatingTest, TestCursorMarkWithoutUniqueKey, HdfsBasicDistributedZk2Test, AlternateDirectoryTest, DistributedExpandComponentTest, ConfigureRecoveryStrategyTest, SOLR749Test, TestFuzzyAnalyzedSuggestions, EnumFieldTest, BaseCdcrDistributedZkTest, TestConfigSetProperties, TestPartialUpdateDeduplication, TestFieldSortValues, SolrSlf4jReporterTest, DistributedIntervalFacetingTest, TestRestManager, SolrCloudExampleTest, DistributedMLTComponentTest, StatsReloadRaceTest, CdcrReplicationHandlerTest, LargeFieldTest, TestSolrCoreSnapshots, BasicFunctionalityTest, MetricsConfigTest, FileUtilsTest, DistributedQueueTest, BadCopyFieldTest, HdfsSyncSliceTest, DistributedFacetPivotWhiteBoxTest, TestCollectionsAPIViaSolrCloudCluster, SolrGraphiteReporterTest, SpellCheckComponentTest, TestNumericTokenStream, TestHashQParserPlugin, TestManagedStopFilterFactory, TestSearchPerf, HLLSerializationTest, TestBlobHandler, DistribCursorPagingTest, ZkControllerTest, HdfsRecoverLeaseTest, TestConfigSetsAPIZkFailure, TestSolrJ, UpdateRequestProcessorFactoryTest]
   [junit4] Completed [221/732 (2!)] on J1 in 4.95s, 3 tests, 1 failure <<< FAILURES!

[...truncated 43938 lines...]
-ecj-javadoc-lint-src:
    [mkdir] Created dir: /tmp/ecj1189216355
 [ecj-lint] Compiling 1124 source files to /tmp/ecj1189216355
 [ecj-lint] invalid Class-Path header in manifest of jar file: /home/jenkins/.ivy2/cache/org.restlet.jee/org.restlet/jars/org.restlet-2.3.0.jar
 [ecj-lint] invalid Class-Path header in manifest of jar file: /home/jenkins/.ivy2/cache/org.restlet.jee/org.restlet.ext.servlet/jars/org.restlet.ext.servlet-2.3.0.jar
 [ecj-lint] ----------
 [ecj-lint] 1. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/core/CoreContainer.java (at line 1002)
 [ecj-lint] core = new SolrCore(this, dcore, coreConfig);
 [ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'core' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 2. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/core/HdfsDirectoryFactory.java (at line 234)
 [ecj-lint] dir = new BlockDirectory(path, hdfsDir, cache, null, blockCacheReadEnabled, false, cacheMerges, cacheReadOnce);
 [ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'dir' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 3. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/handler/AnalysisRequestHandlerBase.java (at line 121)
 [ecj-lint] reader = cfiltfac.create(reader);
 [ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'reader' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] 4. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/handler/AnalysisRequestHandlerBase.java (at line 145)
 [ecj-lint] return namedList;
 [ecj-lint] ^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'listBasedTokenStream' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 5. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/handler/ReplicationHandler.java (at line 1282)
 [ecj-lint] DirectoryReader reader = s==null ? null : s.get().getIndexReader();
 [ecj-lint]                ^^^^^^
 [ecj-lint] Resource leak: 'reader' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 6. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/handler/sql/SolrTable.java (at line 517)
 [ecj-lint] ParallelStream parallelStream = new ParallelStream(zk, collection, tupleStream, numWorkers, comp);
 [ecj-lint]               ^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'parallelStream' is never closed
 [ecj-lint] ----------
 [ecj-lint] 7. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/handler/sql/SolrTable.java (at line 743)
 [ecj-lint] ParallelStream parallelStream = new ParallelStream(zkHost, collection, tupleStream, numWorkers, comp);
 [ecj-lint]               ^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'parallelStream' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 8. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java (at line 578)
 [ecj-lint] tvWindowStream = new OffsetWindowTokenFilter(tvStream);
 [ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'tvWindowStream' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 9. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/metrics/reporters/SolrSlf4jReporter.java (at line 21)
 [ecj-lint] import java.util.SortedMap;
 [ecj-lint]       ^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import java.util.SortedMap is never used
 [ecj-lint] ----------
 [ecj-lint] 10. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/metrics/reporters/SolrSlf4jReporter.java (at line 24)
 [ecj-lint] import com.codahale.metrics.Counter;
 [ecj-lint]       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import com.codahale.metrics.Counter is never used
 [ecj-lint] ----------
 [ecj-lint] 11. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/metrics/reporters/SolrSlf4jReporter.java (at line 25)
 [ecj-lint] import com.codahale.metrics.Gauge;
 [ecj-lint]       ^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import com.codahale.metrics.Gauge is never used
 [ecj-lint] ----------
 [ecj-lint] 12. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/metrics/reporters/SolrSlf4jReporter.java (at line 26)
 [ecj-lint] import com.codahale.metrics.Histogram;
 [ecj-lint]       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import com.codahale.metrics.Histogram is never used
 [ecj-lint] ----------
 [ecj-lint] 13. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/metrics/reporters/SolrSlf4jReporter.java (at line 27)
 [ecj-lint] import com.codahale.metrics.Meter;
 [ecj-lint]       ^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import com.codahale.metrics.Meter is never used
 [ecj-lint] ----------
 [ecj-lint] 14. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/metrics/reporters/SolrSlf4jReporter.java (at line 29)
 [ecj-lint] import com.codahale.metrics.MetricRegistry;
 [ecj-lint]       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import com.codahale.metrics.MetricRegistry is never used
 [ecj-lint] ----------
 [ecj-lint] 15. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/metrics/reporters/SolrSlf4jReporter.java (at line 32)
 [ecj-lint] import com.codahale.metrics.Timer;
 [ecj-lint]       ^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import com.codahale.metrics.Timer is never used
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 16. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/request/SimpleFacets.java (at line 925)
 [ecj-lint] fastForRandomSet = new HashDocSet(sset.getDocs(), 0, sset.size());
 [ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'fastForRandomSet' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 17. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/response/SmileResponseWriter.java (at line 33)
 [ecj-lint] new SmileWriter(out, request, response).writeResponse();
 [ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 18. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/schema/OpenExchangeRatesOrgProvider.java (at line 146)
 [ecj-lint] ratesJsonStream = resourceLoader.openResource(ratesFileLocation);
 [ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'ratesJsonStream' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 19. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/search/CollapsingQParserPlugin.java (at line 1315)
 [ecj-lint] UninvertingReader uninvertingReader = new UninvertingReader(new ReaderWrapper(searcher.getSlowAtomicReader(), collapseField), mapping);
 [ecj-lint]                  ^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'uninvertingReader' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 20. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 145)
 [ecj-lint] HashDocSet h = new HashDocSet(docs,offset,len);
 [ecj-lint]           ^
 [ecj-lint] Resource leak: 'h' is never closed
 [ecj-lint] ----------
 [ecj-lint] 21. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 154)
 [ecj-lint] HashDocSet h = new HashDocSet(docs,offset,len);
 [ecj-lint]           ^
 [ecj-lint] Resource leak: 'h' is never closed
 [ecj-lint] ----------
 [ecj-lint] 22. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 163)
 [ecj-lint] HashDocSet h = new HashDocSet(docs,offset,len);
 [ecj-lint]           ^
 [ecj-lint] Resource leak: 'h' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 23. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/search/function/OrdFieldSource.java (at line 79)
 [ecj-lint] SolrIndexSearcher is = (SolrIndexSearcher) o;
 [ecj-lint]                  ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 24. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/search/function/ReverseOrdFieldSource.java (at line 79)
 [ecj-lint] SolrIndexSearcher is = (SolrIndexSearcher) o;
 [ecj-lint]                  ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 25. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/servlet/SolrDispatchFilter.java (at line 291)
 [ecj-lint] throw new SolrException(ErrorCode.SERVER_ERROR, "Error occurred while loading solr.xml from zookeeper", e);
 [ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'loader' is not closed at this location
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 26. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/update/CdcrTransactionLog.java (at line 98)
 [ecj-lint] ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
 [ecj-lint]                       ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 27. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/update/HdfsTransactionLog.java (at line 191)
 [ecj-lint] LogCodec codec = new LogCodec(resolver);
 [ecj-lint]         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] 28. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/update/HdfsTransactionLog.java (at line 261)
 [ecj-lint] LogCodec codec = new LogCodec(resolver);
 [ecj-lint]         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 29. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 228)
 [ecj-lint] ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
 [ecj-lint]                       ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] 30. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 228)
 [ecj-lint] ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
 [ecj-lint]                       ^^
 [ecj-lint] Resource leak: 'is' is never closed
 [ecj-lint] ----------
 [ecj-lint] 31. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 260)
 [ecj-lint] LogCodec codec = new LogCodec(resolver);
 [ecj-lint]         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] 32. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 275)
 [ecj-lint] LogCodec codec = new LogCodec(resolver);
 [ecj-lint]         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] 33. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 529)
 [ecj-lint] LogCodec codec = new LogCodec(resolver);
 [ecj-lint]         ^^^^^
 [ecj-lint] Resource leak: 'codec' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 34. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/update/processor/UpdateRequestProcessor.java (at line 85)
 [ecj-lint] UpdateRequestProcessor p = this;
 [ecj-lint]                       ^
 [ecj-lint] Resource leak: 'p' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 35. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 655)
 [ecj-lint] BAOS bos =  new BAOS();
 [ecj-lint]     ^^^
 [ecj-lint] Resource leak: 'bos' is never closed
 [ecj-lint] ----------
 [ecj-lint] 36. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 655)
 [ecj-lint] BAOS bos =  new BAOS();
 [ecj-lint]     ^^^
 [ecj-lint] Resource leak: 'bos' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 37. WARNING in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/java/org/apache/solr/util/SolrLogLayout.java (at line 149)
 [ecj-lint] SolrQueryRequest req = requestInfo == null ? null : requestInfo.getReq();
 [ecj-lint]                 ^^^
 [ecj-lint] Resource leak: 'req' is never closed
 [ecj-lint] ----------
 [ecj-lint] 37 problems (7 errors, 30 warnings)

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:810: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:101: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build.xml:689: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:2030: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:2069: Compile failed; see the compiler error output for details.

Total time: 231 minutes 38 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2087 - Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2087/

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderElectionContextKeyTest.test

Error Message:


Stack Trace:
java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([756C6320B1786F0D:FD385CFA1F8402F5]:0)
        at org.junit.Assert.fail(Assert.java:92)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertTrue(Assert.java:54)
        at org.apache.solr.cloud.LeaderElectionContextKeyTest.test(LeaderElectionContextKeyTest.java:88)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
Expected 2 of 3 replicas to be active but only found 1; [core_node5:{"core":"c8n_1x3_lf_shard1_replica_t1","base_url":"http://127.0.0.1:38474/x","node_name":"127.0.0.1:38474_x","state":"active","type":"TLOG","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/16)={   "pullReplicas":"0",   "replicationFactor":"1",   "shards":{"shard1":{       "range":"80000000-7fffffff",       "state":"active",       "replicas":{         "core_node4":{           "state":"down",           "base_url":"http://127.0.0.1:45700/x",           "core":"c8n_1x3_lf_shard1_replica_t3",           "node_name":"127.0.0.1:45700_x",           "type":"TLOG"},         "core_node5":{           "core":"c8n_1x3_lf_shard1_replica_t1",           "base_url":"http://127.0.0.1:38474/x",           "node_name":"127.0.0.1:38474_x",           "state":"active",           "type":"TLOG",           "leader":"true"},         "core_node6":{           "core":"c8n_1x3_lf_shard1_replica_t2",           "base_url":"http://127.0.0.1:39645/x",           "node_name":"127.0.0.1:39645_x",           "state":"down",           "type":"TLOG"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"1",   "autoAddReplicas":"false",   "nrtReplicas":"0",   "tlogReplicas":"3"}

Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 1; [core_node5:{"core":"c8n_1x3_lf_shard1_replica_t1","base_url":"http://127.0.0.1:38474/x","node_name":"127.0.0.1:38474_x","state":"active","type":"TLOG","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/16)={
  "pullReplicas":"0",
  "replicationFactor":"1",
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{
        "core_node4":{
          "state":"down",
          "base_url":"http://127.0.0.1:45700/x",
          "core":"c8n_1x3_lf_shard1_replica_t3",
          "node_name":"127.0.0.1:45700_x",
          "type":"TLOG"},
        "core_node5":{
          "core":"c8n_1x3_lf_shard1_replica_t1",
          "base_url":"http://127.0.0.1:38474/x",
          "node_name":"127.0.0.1:38474_x",
          "state":"active",
          "type":"TLOG",
          "leader":"true"},
        "core_node6":{
          "core":"c8n_1x3_lf_shard1_replica_t2",
          "base_url":"http://127.0.0.1:39645/x",
          "node_name":"127.0.0.1:39645_x",
          "state":"down",
          "type":"TLOG"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"0",
  "tlogReplicas":"3"}
        at __randomizedtesting.SeedInfo.seed([756C6320B1786F0D:FD385CFA1F8402F5]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:169)
        at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:56)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 11514 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderElectionContextKeyTest
   [junit4]   2> 784487 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderElectionContextKeyTest_756C6320B1786F0D-001/init-core-data-001
   [junit4]   2> 784488 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 784488 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 784501 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderElectionContextKeyTest_756C6320B1786F0D-001/tempDir-001
   [junit4]   2> 784501 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 784501 INFO  (Thread-439) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 784501 INFO  (Thread-439) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 784502 ERROR (Thread-439) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 784663 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:46732
   [junit4]   2> 784818 INFO  (jetty-launcher-340-thread-1) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 784879 INFO  (jetty-launcher-340-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6f90bb4e{/solr,null,AVAILABLE}
   [junit4]   2> 784880 INFO  (jetty-launcher-340-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@15d1cb98{HTTP/1.1,[http/1.1]}{127.0.0.1:36032}
   [junit4]   2> 784880 INFO  (jetty-launcher-340-thread-1) [    ] o.e.j.s.Server Started @797249ms
   [junit4]   2> 784880 INFO  (jetty-launcher-340-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36032}
   [junit4]   2> 784880 ERROR (jetty-launcher-340-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 784881 INFO  (jetty-launcher-340-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 784881 INFO  (jetty-launcher-340-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 784881 INFO  (jetty-launcher-340-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 784881 INFO  (jetty-launcher-340-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-03T12:09:04.114Z
   [junit4]   2> 784968 INFO  (jetty-launcher-340-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 785009 INFO  (jetty-launcher-340-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46732/solr
   [junit4]   2> 785492 INFO  (jetty-launcher-340-thread-1) [n:127.0.0.1:36032_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 785492 INFO  (jetty-launcher-340-thread-1) [n:127.0.0.1:36032_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36032_solr
   [junit4]   2> 785492 INFO  (jetty-launcher-340-thread-1) [n:127.0.0.1:36032_solr    ] o.a.s.c.Overseer Overseer (id=98595015474216963-127.0.0.1:36032_solr-n_0000000000) starting
   [junit4]   2> 785598 INFO  (jetty-launcher-340-thread-1) [n:127.0.0.1:36032_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36032_solr
   [junit4]   2> 785598 INFO  (zkCallback-347-thread-1-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 786257 INFO  (jetty-launcher-340-thread-1) [n:127.0.0.1:36032_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36032.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f0e16d8
   [junit4]   2> 786353 INFO  (jetty-launcher-340-thread-1) [n:127.0.0.1:36032_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36032.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f0e16d8
   [junit4]   2> 786353 INFO  (jetty-launcher-340-thread-1) [n:127.0.0.1:36032_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36032.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f0e16d8
   [junit4]   2> 786354 INFO  (jetty-launcher-340-thread-1) [n:127.0.0.1:36032_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderElectionContextKeyTest_756C6320B1786F0D-001/tempDir-001/node1/.
   [junit4]   2> 786718 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 786719 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46732/solr ready
   [junit4]   2> 786755 INFO  (qtp2027561654-2652) [n:127.0.0.1:36032_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=config&maxShardsPerNode=1000&name=testCollection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 786864 INFO  (OverseerThreadFactory-1189-thread-1-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.CreateCollectionCmd Create collection testCollection1
   [junit4]   2> 787048 INFO  (OverseerStateUpdate-98595015474216963-127.0.0.1:36032_solr-n_0000000000) [n:127.0.0.1:36032_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testCollection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testCollection1_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:36032/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 787111 INFO  (OverseerStateUpdate-98595015474216963-127.0.0.1:36032_solr-n_0000000000) [n:127.0.0.1:36032_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testCollection1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testCollection1_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:36032/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 787322 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=config&newCollection=true&name=testCollection1_shard1_replica_n1&action=CREATE&numShards=2&collection=testCollection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 787323 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 787327 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=testCollection1_shard2_replica_n2&action=CREATE&numShards=2&collection=testCollection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 787464 INFO  (zkCallback-347-thread-1-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 787465 INFO  (zkCallback-347-thread-2-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 788392 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 788411 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 788411 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.s.IndexSchema [testCollection1_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 788426 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 788426 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testCollection1_shard2_replica_n2' using configuration from collection testCollection1, trusted=true
   [junit4]   2> 788427 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36032.solr.core.testCollection1.shard2.replica_n2' (registry 'solr.core.testCollection1.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f0e16d8
   [junit4]   2> 788427 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 788427 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.SolrCore [[testCollection1_shard2_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderElectionContextKeyTest_756C6320B1786F0D-001/tempDir-001/node1/testCollection1_shard2_replica_n2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderElectionContextKeyTest_756C6320B1786F0D-001/tempDir-001/node1/./testCollection1_shard2_replica_n2/data/]
   [junit4]   2> 788443 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.s.IndexSchema [testCollection1_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 788474 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 788474 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testCollection1_shard1_replica_n1' using configuration from collection testCollection1, trusted=true
   [junit4]   2> 788475 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36032.solr.core.testCollection1.shard1.replica_n1' (registry 'solr.core.testCollection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f0e16d8
   [junit4]   2> 788475 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 788475 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrCore [[testCollection1_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderElectionContextKeyTest_756C6320B1786F0D-001/tempDir-001/node1/testCollection1_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderElectionContextKeyTest_756C6320B1786F0D-001/tempDir-001/node1/./testCollection1_shard1_replica_n1/data/]
   [junit4]   2> 788621 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 788621 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 788622 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 788622 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 788623 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@576e7ef[testCollection1_shard1_replica_n1] main]
   [junit4]   2> 788624 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 788624 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 788625 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 788625 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 788626 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@74e525f7[testCollection1_shard2_replica_n2] main]
   [junit4]   2> 788637 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
   [junit4]   2> 788637 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
   [junit4]   2> 788637 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 788637 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 788638 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 788638 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 788638 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577520251924381696
   [junit4]   2> 788638 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577520251924381696
   [junit4]   2> 788653 INFO  (searcherExecutor-1192-thread-1-processing-n:127.0.0.1:36032_solr x:testCollection1_shard2_replica_n2 s:shard2 c:testCollection1 r:core_node4) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.SolrCore [testCollection1_shard2_replica_n2] Registered new searcher Searcher@74e525f7[testCollection1_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 788654 INFO  (searcherExecutor-1193-thread-1-processing-n:127.0.0.1:36032_solr x:testCollection1_shard1_replica_n1 s:shard1 c:testCollection1 r:core_node3) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrCore [testCollection1_shard1_replica_n1] Registered new searcher Searcher@576e7ef[testCollection1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 788669 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 788669 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 788669 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36032/solr/testCollection1_shard2_replica_n2/
   [junit4]   2> 788669 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 788669 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:36032/solr/testCollection1_shard2_replica_n2/ has no replicas
   [junit4]   2> 788669 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 788683 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36032/solr/testCollection1_shard2_replica_n2/ shard2
   [junit4]   2> 788683 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 788683 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 788683 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36032/solr/testCollection1_shard1_replica_n1/
   [junit4]   2> 788683 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 788683 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:36032/solr/testCollection1_shard1_replica_n1/ has no replicas
   [junit4]   2> 788683 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 788698 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36032/solr/testCollection1_shard1_replica_n1/ shard1
   [junit4]   2> 788800 INFO  (zkCallback-347-thread-2-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 788800 INFO  (zkCallback-347-thread-1-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 788834 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 788850 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 788880 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=testCollection1_shard2_replica_n2&action=CREATE&numShards=2&collection=testCollection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1552
   [junit4]   2> 788881 INFO  (qtp2027561654-2653) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=config&newCollection=true&name=testCollection1_shard1_replica_n1&action=CREATE&numShards=2&collection=testCollection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1558
   [junit4]   2> 788992 INFO  (zkCallback-347-thread-1-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 788993 INFO  (zkCallback-347-thread-2-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 789011 INFO  (qtp2027561654-2652) [n:127.0.0.1:36032_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 789011 INFO  (qtp2027561654-2652) [n:127.0.0.1:36032_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=config&maxShardsPerNode=1000&name=testCollection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2256
   [junit4]   2> 789018 INFO  (qtp2027561654-2655) [n:127.0.0.1:36032_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=config&maxShardsPerNode=1000&name=testCollection2&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 789020 INFO  (OverseerCollectionConfigSetProcessor-98595015474216963-127.0.0.1:36032_solr-n_0000000000) [n:127.0.0.1:36032_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 789021 INFO  (OverseerThreadFactory-1189-thread-2-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.CreateCollectionCmd Create collection testCollection2
   [junit4]   2> 789237 INFO  (OverseerStateUpdate-98595015474216963-127.0.0.1:36032_solr-n_0000000000) [n:127.0.0.1:36032_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testCollection2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testCollection2_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:36032/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 789239 INFO  (OverseerStateUpdate-98595015474216963-127.0.0.1:36032_solr-n_0000000000) [n:127.0.0.1:36032_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testCollection2",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testCollection2_shard2_replica_n3",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:36032/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 789379 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=testCollection2_shard2_replica_n3&action=CREATE&numShards=2&collection=testCollection2&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 789380 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=config&newCollection=true&name=testCollection2_shard1_replica_n1&action=CREATE&numShards=2&collection=testCollection2&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 789562 INFO  (zkCallback-347-thread-2-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 789562 INFO  (zkCallback-347-thread-1-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 790454 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 790611 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.s.IndexSchema [testCollection2_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 790625 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 790625 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testCollection2_shard1_replica_n1' using configuration from collection testCollection2, trusted=true
   [junit4]   2> 790626 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 790626 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36032.solr.core.testCollection2.shard1.replica_n1' (registry 'solr.core.testCollection2.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f0e16d8
   [junit4]   2> 790626 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 790626 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrCore [[testCollection2_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderElectionContextKeyTest_756C6320B1786F0D-001/tempDir-001/node1/testCollection2_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderElectionContextKeyTest_756C6320B1786F0D-001/tempDir-001/node1/./testCollection2_shard1_replica_n1/data/]
   [junit4]   2> 790674 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.s.IndexSchema [testCollection2_shard2_replica_n3] Schema name=minimal
   [junit4]   2> 790676 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 790677 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'testCollection2_shard2_replica_n3' using configuration from collection testCollection2, trusted=true
   [junit4]   2> 790677 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36032.solr.core.testCollection2.shard2.replica_n3' (registry 'solr.core.testCollection2.shard2.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f0e16d8
   [junit4]   2> 790690 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 790690 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SolrCore [[testCollection2_shard2_replica_n3] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderElectionContextKeyTest_756C6320B1786F0D-001/tempDir-001/node1/testCollection2_shard2_replica_n3], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderElectionContextKeyTest_756C6320B1786F0D-001/tempDir-001/node1/./testCollection2_shard2_replica_n3/data/]
   [junit4]   2> 790806 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 790806 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 790807 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 790807 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 790821 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3dccfc9c[testCollection2_shard1_replica_n1] main]
   [junit4]   2> 790876 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
   [junit4]   2> 790892 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 790893 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 790894 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577520254289969152
   [junit4]   2> 790895 INFO  (searcherExecutor-1202-thread-1-processing-n:127.0.0.1:36032_solr x:testCollection2_shard1_replica_n1 s:shard1 c:testCollection2 r:core_node2) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrCore [testCollection2_shard1_replica_n1] Registered new searcher Searcher@3dccfc9c[testCollection2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 790899 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 790899 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 790899 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36032/solr/testCollection2_shard1_replica_n1/
   [junit4]   2> 790899 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 790899 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:36032/solr/testCollection2_shard1_replica_n1/ has no replicas
   [junit4]   2> 790899 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 790921 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 790921 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 790922 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 790922 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 790924 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@7c284740[testCollection2_shard2_replica_n3] main]
   [junit4]   2> 790938 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36032/solr/testCollection2_shard1_replica_n1/ shard1
   [junit4]   2> 790938 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
   [junit4]   2> 790940 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 790940 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 790974 INFO  (searcherExecutor-1203-thread-1-processing-n:127.0.0.1:36032_solr x:testCollection2_shard2_replica_n3 s:shard2 c:testCollection2 r:core_node4) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SolrCore [testCollection2_shard2_replica_n3] Registered new searcher Searcher@7c284740[testCollection2_shard2_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 790977 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577520254377000960
   [junit4]   2> 791007 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 791007 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 791007 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36032/solr/testCollection2_shard2_replica_n3/
   [junit4]   2> 791007 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 791007 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SyncStrategy http://127.0.0.1:36032/solr/testCollection2_shard2_replica_n3/ has no replicas
   [junit4]   2> 791007 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 791009 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36032/solr/testCollection2_shard2_replica_n3/ shard2
   [junit4]   2> 791237 INFO  (zkCallback-347-thread-1-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 791237 INFO  (zkCallback-347-thread-2-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 791259 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 791260 INFO  (qtp2027561654-2658) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=testCollection2_shard2_replica_n3&action=CREATE&numShards=2&collection=testCollection2&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1880
   [junit4]   2> 791290 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 791371 INFO  (qtp2027561654-2657) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=config&newCollection=true&name=testCollection2_shard1_replica_n1&action=CREATE&numShards=2&collection=testCollection2&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1991
   [junit4]   2> 791389 INFO  (qtp2027561654-2655) [n:127.0.0.1:36032_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 791533 INFO  (zkCallback-347-thread-2-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 791533 INFO  (zkCallback-347-thread-1-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 792402 INFO  (qtp2027561654-2655) [n:127.0.0.1:36032_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=config&maxShardsPerNode=1000&name=testCollection2&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=3384
   [junit4]   2> 792402 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: testCollection1 failOnTimeout:true timeout (sec):30
   [junit4]   2> 792418 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: testCollection1
   [junit4]   2> 792418 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: testCollection2 failOnTimeout:true timeout (sec):30
   [junit4]   2> 792434 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: testCollection2
   [junit4]   2> 792464 INFO  (TEST-LeaderElectionContextKeyTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 792568 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection1.shard2.replica_n2, tag=null
   [junit4]   2> 792568 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2b3c911c: rootName = solr_36032, domain = solr.core.testCollection1.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.testCollection1.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@a8bb22e
   [junit4]   2> 792682 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr    ] o.a.s.c.SolrCore [testCollection1_shard2_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@7b0d09d8
   [junit4]   2> 792682 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection1.shard2.replica_n2, tag=2064452056
   [junit4]   2> 792682 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection1.shard2.leader, tag=2064452056
   [junit4]   2> 792728 INFO  (qtp2027561654-2656) [n:127.0.0.1:36032_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=false&core=testCollection1_shard2_replica_n2&deleteDataDir=false&action=UNLOAD&indexInfo=true&wt=javabin&version=2&deleteIndex=false} status=0 QTime=176
   [junit4]   2> 792729 INFO  (TEST-LeaderElectionContextKeyTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.SolrTestCaseJ4 ###Ending test
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderElectionContextKeyTest -Dtests.method=test -Dtests.seed=756C6320B1786F0D -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=sr-BA -Dtests.timezone=Africa/Bujumbura -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.31s J1 | LeaderElectionContextKeyTest.test <<<
   [junit4]    > Throwable #1: java.lang.AssertionError
   [junit4]    > at __randomizedtesting.SeedInfo.seed([756C6320B1786F0D:FD385CFA1F8402F5]:0)
   [junit4]    > at org.apache.solr.cloud.LeaderElectionContextKeyTest.test(LeaderElectionContextKeyTest.java:88)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 792780 INFO  (jetty-closer-341-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@15d1cb98{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 792781 INFO  (jetty-closer-341-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=397067939
   [junit4]   2> 792781 INFO  (jetty-closer-341-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 792781 INFO  (jetty-closer-341-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@14dd20f2: rootName = solr_36032, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7f730b9e
   [junit4]   2> 792785 INFO  (jetty-closer-341-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 792785 INFO  (jetty-closer-341-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1bf31089: rootName = solr_36032, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@65a5129b
   [junit4]   2> 792804 INFO  (jetty-closer-341-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 792804 INFO  (jetty-closer-341-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@56d96292: rootName = solr_36032, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@619a9af7
   [junit4]   2> 792817 INFO  (jetty-closer-341-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 792818 INFO  (coreCloseExecutor-1212-thread-1) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrCore [testCollection1_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@6fef0b6a
   [junit4]   2> 792818 INFO  (coreCloseExecutor-1212-thread-1) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection1.shard1.replica_n1, tag=1877937002
   [junit4]   2> 792818 INFO  (coreCloseExecutor-1212-thread-1) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3a74a40f: rootName = solr_36032, domain = solr.core.testCollection1.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.testCollection1.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@4a9c943
   [junit4]   2> 792848 INFO  (coreCloseExecutor-1212-thread-2) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrCore [testCollection2_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@561151ff
   [junit4]   2> 792848 INFO  (coreCloseExecutor-1212-thread-3) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SolrCore [testCollection2_shard2_replica_n3]  CLOSING SolrCore org.apache.solr.core.SolrCore@2c376c6d
   [junit4]   2> 792882 INFO  (coreCloseExecutor-1212-thread-2) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection2.shard1.replica_n1, tag=1443975679
   [junit4]   2> 792882 INFO  (coreCloseExecutor-1212-thread-2) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5351aeb1: rootName = solr_36032, domain = solr.core.testCollection2.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.testCollection2.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@12a2431f
   [junit4]   2> 792900 INFO  (coreCloseExecutor-1212-thread-2) [n:127.0.0.1:36032_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection2.shard1.leader, tag=1443975679
   [junit4]   2> 792905 INFO  (coreCloseExecutor-1212-thread-3) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection2.shard2.replica_n3, tag=741829741
   [junit4]   2> 792905 INFO  (coreCloseExecutor-1212-thread-3) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6382f881: rootName = solr_36032, domain = solr.core.testCollection2.shard2.replica_n3, service url = null, agent id = null] for registry solr.core.testCollection2.shard2.replica_n3 / com.codahale.metrics.MetricRegistry@b8a7791
   [junit4]   2> 792949 INFO  (coreCloseExecutor-1212-thread-3) [n:127.0.0.1:36032_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection2.shard2.leader, tag=741829741
   [junit4]   2> 792949 INFO  (coreCloseExecutor-1212-thread-1) [n:127.0.0.1:36032_solr c:testCollection1 s:shard1 r:core_node3 x:testCollection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection1.shard1.leader, tag=1877937002
   [junit4]   2> 792995 INFO  (jetty-closer-341-thread-1) [    ] o.a.s.c.Overseer Overseer (id=98595015474216963-127.0.0.1:36032_solr-n_0000000000) closing
   [junit4]   2> 792995 INFO  (zkCallback-347-thread-1-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 792995 INFO  (OverseerStateUpdate-98595015474216963-127.0.0.1:36032_solr-n_0000000000) [n:127.0.0.1:36032_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36032_solr
   [junit4]   2> 793007 INFO  (zkCallback-347-thread-2-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 794511 WARN  (zkCallback-347-thread-2-processing-n:127.0.0.1:36032_solr) [n:127.0.0.1:36032_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 794512 INFO  (jetty-closer-341-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6f90bb4e{/solr,null,UNAVAILABLE}
   [junit4]   2> 794512 ERROR (SUITE-LeaderElectionContextKeyTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 794513 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46732 46732
   [junit4]   2> 794683 INFO  (Thread-439) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46732 46732
   [junit4]   2> 794684 WARN  (Thread-439) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 2 /solr/aliases.json
   [junit4]   2> 2 /solr/clusterprops.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 6 /solr/collections/testCollection1/state.json
   [junit4]   2> 5 /solr/collections/testCollection2/state.json
   [junit4]   2> 2 /solr/clusterstate.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 2 /solr/live_nodes
   [junit4]   2> 2 /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderElectionContextKeyTest_756C6320B1786F0D-001
   [junit4]   2> Sep 03, 2017 12:09:13 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), sim=RandomSimilarity(queryNorm=true): {}, locale=sr-BA, timezone=Africa/Bujumbura
   [junit4]   2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=110105136,total=460324864
   [junit4]   2> NOTE: All tests run in this JVM: [FileUtilsTest, BitVectorTest, DistanceFunctionTest, TestSubQueryTransformerDistrib, TestTlogReplica, ConfigureRecoveryStrategyTest, TestSystemIdResolver, TestAtomicUpdateErrorCases, TestConfigSetProperties, ChaosMonkeyNothingIsSafeWithPullReplicasTest, ChaosMonkeySafeLeaderTest, SolrCmdDistributorTest, TestExpandComponent, ImplicitSnitchTest, OutOfBoxZkACLAndCredentialsProvidersTest, ForceLeaderTest, BasicFunctionalityTest, LeaderInitiatedRecoveryOnCommitTest, TestCursorMarkWithoutUniqueKey, AtomicUpdatesTest, CurrencyFieldTypeTest, JsonLoaderTest, MetricsConfigTest, LeaderElectionContextKeyTest]
   [junit4] Completed [74/732 (1!)] on J1 in 10.57s, 1 test, 1 failure <<< FAILURES!

[...truncated 1519 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_756C6320B1786F0D-001/init-core-data-001
   [junit4]   2> 4274766 WARN  (SUITE-LeaderFailoverAfterPartitionTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=42 numCloses=42
   [junit4]   2> 4274766 INFO  (SUITE-LeaderFailoverAfterPartitionTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 4274767 INFO  (SUITE-LeaderFailoverAfterPartitionTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 4274767 INFO  (SUITE-LeaderFailoverAfterPartitionTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 4274767 INFO  (SUITE-LeaderFailoverAfterPartitionTest-seed#[756C6320B1786F0D]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /x/
   [junit4]   2> 4274796 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 4274800 INFO  (Thread-3607) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 4274800 INFO  (Thread-3607) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 4274812 ERROR (Thread-3607) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 4274906 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.ZkTestServer start zk server on port:35636
   [junit4]   2> 4275075 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 4275075 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 4275105 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4275106 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 4275107 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 4275107 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 4275108 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 4275109 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4275109 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4275110 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 4275111 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 4275111 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e47da2d8d0001, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 4275112 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
   [junit4]   2> 4275440 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 4275440 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@24e2908a{/x,null,AVAILABLE}
   [junit4]   2> 4275441 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@623bcd12{HTTP/1.1,[http/1.1]}{127.0.0.1:39055}
   [junit4]   2> 4275442 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.e.j.s.Server Started @4288217ms
   [junit4]   2> 4275442 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_756C6320B1786F0D-001/tempDir-001/control/data, replicaType=NRT, hostContext=/x, hostPort=45700, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_756C6320B1786F0D-001/control-001/cores}
   [junit4]   2> 4275454 ERROR (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 4275454 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 4275454 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 4275454 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 4275455 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-03T13:07:15.088Z
   [junit4]   2> 4275468 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 4275468 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_756C6320B1786F0D-001/control-001/solr.xml
   [junit4]   2> 4275485 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@35975bfe, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 4275487 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35636/solr
   [junit4]   2> 4275664 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [n:127.0.0.1:45700_x    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 4275665 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [n:127.0.0.1:45700_x    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45700_x
   [junit4]   2> 4275665 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [n:127.0.0.1:45700_x    ] o.a.s.c.Overseer Overseer (id=98595244241387524-127.0.0.1:45700_x-n_0000000000) starting
   [junit4]   2> 4275747 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [n:127.0.0.1:45700_x    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45700_x
   [junit4]   2> 4275767 INFO  (zkCallback-2843-thread-1-processing-n:127.0.0.1:45700_x) [n:127.0.0.1:45700_x    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 4276042 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [n:127.0.0.1:45700_x    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@35975bfe
   [junit4]   2> 4276071 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [n:127.0.0.1:45700_x    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@35975bfe
   [junit4]   2> 4276071 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [n:127.0.0.1:45700_x    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@35975bfe
   [junit4]   2> 4276080 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [n:127.0.0.1:45700_x    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_756C6320B1786F0D-001/control-001/cores
   [junit4]   2> 4276148 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 4276149 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35636/solr ready
   [junit4]   2> 4276149 INFO  (SocketProxy-Acceptor-45700) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=41454,localport=45700], receiveBufferSize:531000
   [junit4]   2> 4276150 INFO  (SocketProxy-Acceptor-45700) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39055,localport=50026], receiveBufferSize=530904
   [junit4]   2> 4276151 INFO  (qtp1434484385-16640) [n:127.0.0.1:45700_x    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:45700_x&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 4276153 INFO  (OverseerThreadFactory-6834-thread-1-processing-n:127.0.0.1:45700_x) [n:127.0.0.1:45700_x    ] o.a.s.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 4276270 INFO  (SocketProxy-Acceptor-45700) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=41458,localport=45700], receiveBufferSize:531000
   [junit4]   2> 4276286 INFO  (SocketProxy-Acceptor-45700) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39055,localport=50030], receiveBufferSize=530904
   [junit4]   2> 4276290 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 4276291 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 4276453 INFO  (zkCallback-2843-thread-2-processing-n:127.0.0.1:45700_x) [n:127.0.0.1:45700_x    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 4277423 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 4277498 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 4277962 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 4278010 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 4278011 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@35975bfe
   [junit4]   2> 4278011 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 4278011 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_756C6320B1786F0D-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_756C6320B1786F0D-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 4278013 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=17, maxMergeAtOnceExplicit=35, maxMergedSegmentMB=45.517578125, floorSegmentMB=1.859375, forceMergeDeletesPctAllowed=6.362754710662157, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.36362491534065977
   [junit4]   2> 4278016 WARN  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 4278126 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 4278126 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 4278143 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 4278144 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 4278145 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=28, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.37336106787874856]
   [junit4]   2> 4278145 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5de469eb[control_collection_shard1_replica_n1] main]
   [junit4]   2> 4278146 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 4278146 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4278146 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 4278147 INFO  (searcherExecutor-6837-thread-1-processing-n:127.0.0.1:45700_x x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@5de469eb[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 4278155 INFO  (qtp1434484385-16638) [n:127.0.0.1:45700_x c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577523911367589888
   [junit4]   2> 4278168 INF

[...truncated too long message...]


   [junit4]   2> 6 /solr/clusterprops.json
   [junit4]   2> 6 /solr/clusterstate.json
   [junit4]   2> 3 /solr/collections/collection1/state.json
   [junit4]   2> 3 /solr/collections/c8n_1x3_lf/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 6 /solr/live_nodes
   [junit4]   2> 6 /solr/collections
   [junit4]   2> 2 /solr/overseer/queue
   [junit4]   2> 2 /solr/overseer/collection-queue-work
   [junit4]   2> 2 /solr/overseer/queue-work
   [junit4]   2>
   [junit4]   2> 4418739 WARN  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.SocketProxy Closing 0 connections to: http://127.0.0.1:39645/x, target: http://127.0.0.1:38468/x
   [junit4]   2> 4418739 WARN  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.SocketProxy Closing 1 connections to: http://127.0.0.1:45700/x, target: http://127.0.0.1:39055/x
   [junit4]   2> 4418739 WARN  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.SocketProxy Closing 4 connections to: http://127.0.0.1:36321/x, target: http://127.0.0.1:34341/x
   [junit4]   2> 4418739 WARN  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[756C6320B1786F0D]) [    ] o.a.s.c.SocketProxy Closing 6 connections to: http://127.0.0.1:38474/x, target: http://127.0.0.1:33269/x
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderFailoverAfterPartitionTest -Dtests.method=test -Dtests.seed=756C6320B1786F0D -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=ca-ES -Dtests.timezone=Asia/Pyongyang -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE  144s J2 | LeaderFailoverAfterPartitionTest.test <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 1; [core_node5:{"core":"c8n_1x3_lf_shard1_replica_t1","base_url":"http://127.0.0.1:38474/x","node_name":"127.0.0.1:38474_x","state":"active","type":"TLOG","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/16)={
   [junit4]    >   "pullReplicas":"0",
   [junit4]    >   "replicationFactor":"1",
   [junit4]    >   "shards":{"shard1":{
   [junit4]    >       "range":"80000000-7fffffff",
   [junit4]    >       "state":"active",
   [junit4]    >       "replicas":{
   [junit4]    >         "core_node4":{
   [junit4]    >           "state":"down",
   [junit4]    >           "base_url":"http://127.0.0.1:45700/x",
   [junit4]    >           "core":"c8n_1x3_lf_shard1_replica_t3",
   [junit4]    >           "node_name":"127.0.0.1:45700_x",
   [junit4]    >           "type":"TLOG"},
   [junit4]    >         "core_node5":{
   [junit4]    >           "core":"c8n_1x3_lf_shard1_replica_t1",
   [junit4]    >           "base_url":"http://127.0.0.1:38474/x",
   [junit4]    >           "node_name":"127.0.0.1:38474_x",
   [junit4]    >           "state":"active",
   [junit4]    >           "type":"TLOG",
   [junit4]    >           "leader":"true"},
   [junit4]    >         "core_node6":{
   [junit4]    >           "core":"c8n_1x3_lf_shard1_replica_t2",
   [junit4]    >           "base_url":"http://127.0.0.1:39645/x",
   [junit4]    >           "node_name":"127.0.0.1:39645_x",
   [junit4]    >           "state":"down",
   [junit4]    >           "type":"TLOG"}}}},
   [junit4]    >   "router":{"name":"compositeId"},
   [junit4]    >   "maxShardsPerNode":"1",
   [junit4]    >   "autoAddReplicas":"false",
   [junit4]    >   "nrtReplicas":"0",
   [junit4]    >   "tlogReplicas":"3"}
   [junit4]    > at __randomizedtesting.SeedInfo.seed([756C6320B1786F0D:FD385CFA1F8402F5]:0)
   [junit4]    > at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:169)
   [junit4]    > at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:56)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_756C6320B1786F0D-001
   [junit4]   2> Sep 03, 2017 1:09:38 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {multiDefault=PostingsFormat(name=Memory), a_t=PostingsFormat(name=Memory), id=Lucene50(blocksize=128), text=Lucene50(blocksize=128)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene70), _version_=DocValuesFormat(name=Memory), intDefault=DocValuesFormat(name=Memory), id_i1=DocValuesFormat(name=Lucene70), range_facet_i_dv=DocValuesFormat(name=Memory), intDvoDefault=DocValuesFormat(name=Asserting), range_facet_l=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=553, maxMBSortInHeap=6.702538825371585, sim=RandomSimilarity(queryNorm=true): {}, locale=ca-ES, timezone=Asia/Pyongyang
   [junit4]   2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=109319176,total=525336576
   [junit4]   2> NOTE: All tests run in this JVM: [DistributedFacetPivotSmallAdvancedTest, MultiTermTest, SpatialRPTFieldTypeTest, TestCryptoKeys, TestDownShardTolerantSearch, TestEmbeddedSolrServerSchemaAPI, TestReRankQParserPlugin, XsltUpdateRequestHandlerTest, ConcurrentDeleteAndCreateCollectionTest, AutoCommitTest, TestIndexSearcher, TestScoreJoinQPScore, TestSubQueryTransformer, AutoScalingHandlerTest, SimplePostToolTest, TestInPlaceUpdatesStandalone, TestManagedSynonymFilterFactory, SuggestComponentTest, UnloadDistributedZkTest, PeerSyncWithIndexFingerprintCachingTest, DeleteShardTest, SolrTestCaseJ4Test, DateRangeFieldTest, RegexBoostProcessorTest, RollingRestartTest, UniqFieldsUpdateProcessorFactoryTest, NumericFieldsTest, TestConfigSets, TestTrie, TestPKIAuthenticationPlugin, TestConfigReload, URLClassifyProcessorTest, TestAddFieldRealTimeGet, TestFastWriter, TestTolerantSearch, SolrJmxReporterCloudTest, TestCloudInspectUtil, CdcrBootstrapTest, TestMissingGroups, ClassificationUpdateProcessorFactoryTest, SolrInfoBeanTest, TestPseudoReturnFields, TestShortCircuitedRequests, TestSerializedLuceneMatchVersion, CoreSorterTest, TestSolrCoreProperties, TestCloudPseudoReturnFields, TermVectorComponentDistributedTest, DeleteLastCustomShardedReplicaTest, TestInitQParser, ResponseLogComponentTest, ShufflingReplicaListTransformerTest, TestNumericRangeQuery32, TestCodecSupport, TestPointFields, TermsComponentTest, PrimitiveFieldTypeTest, TestPullReplicaErrorHandling, TestDynamicFieldResource, TestSuggestSpellingConverter, ChaosMonkeySafeLeaderWithPullReplicasTest, ActionThrottleTest, MergeStrategyTest, BigEndianAscendingWordDeserializerTest, TestAnalyzedSuggestions, TestOverriddenPrefixQueryForCustomFieldType, PathHierarchyTokenizerFactoryTest, MoveReplicaHDFSTest, HdfsThreadLeakTest, TestCloudSchemaless, TestMacroExpander, BasicDistributedZk2Test, TestJsonFacets, FullSolrCloudDistribCmdsTest, TestStressReorder, InfoHandlerTest, AnalyticsQueryTest, HLLUtilTest, TestSurroundQueryParser, LeaderElectionTest, SolrCoreMetricManagerTest, SignatureUpdateProcessorFactoryTest, TriLevelCompositeIdRoutingTest, TestSolrXml, HdfsCollectionsAPIDistributedZkTest, DirectUpdateHandlerTest, SuggesterFSTTest, BufferStoreTest, ZkStateReaderTest, TestSortByMinMaxFunction, TestComplexPhraseQParserPlugin, SortByFunctionTest, FacetPivotSmallTest, TestUseDocValuesAsStored, SimpleFacetsTest, TestSolrCloudWithKerberosAlt, TestQueryWrapperFilter, TemplateUpdateProcessorTest, TestRecovery, CloudExitableDirectoryReaderTest, TestCloudNestedDocsSort, TestLeaderElectionWithEmptyReplica, TestGroupingSearch, DistributedQueryComponentCustomSortTest, SpellPossibilityIteratorTest, TestFieldCacheSort, TestTolerantUpdateProcessorCloud, TestExceedMaxTermLength, TestRawTransformer, DocumentBuilderTest, SolrCoreCheckLockOnStartupTest, TestReversedWildcardFilterFactory, ManagedSchemaRoundRobinCloudTest, CollectionTooManyReplicasTest, TimeZoneUtilsTest, TestTestInjection, SolrXmlInZkTest, TestClusterProperties, ConjunctionSolrSpellCheckerTest, OverseerTest, LeaderElectionIntegrationTest, ShardRoutingTest, RecoveryZkTest, TestReplicationHandler, ClusterStateUpdateTest, TestRandomFaceting, TestZkChroot, ZkCLITest, ShardRoutingCustomTest, TestDistributedSearch, HardAutoCommitTest, DistributedTermsComponentTest, BadIndexSchemaTest, TestSort, TestFiltering, ShowFileRequestHandlerTest, TestCoreDiscovery, CoreAdminHandlerTest, SuggesterTest, SuggesterWFSTTest, SchemaVersionSpecificBehaviorTest, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, DocumentAnalysisRequestHandlerTest, FileBasedSpellCheckerTest, RequestHandlersTest, MoreLikeThisHandlerTest, TestSolrQueryParser, IndexSchemaRuntimeFieldTest, TestCollationField, ReturnFieldsTest, MBeansHandlerTest, TestLFUCache, PreAnalyzedFieldTest, PrimUtilsTest, DateFieldTest, SpellingQueryConverterTest, DOMUtilTest, ClusterStateTest, TestUtils, TestDocumentBuilder, SliceStateTest, SystemInfoHandlerTest, CircularListTest, TestDistributedMissingSort, AssignTest, CollectionReloadTest, CreateCollectionCleanupTest, DeleteInactiveReplicaTest, DeleteReplicaTest, DistribDocExpirationUpdateProcessorTest, DistribJoinFromCollectionTest, DocValuesNotIndexedTest, LeaderFailoverAfterPartitionTest]
   [junit4] Completed [511/732 (2!)] on J2 in 144.81s, 1 test, 1 failure <<< FAILURES!

[...truncated 46812 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2088 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2088/

2 tests failed.
FAILED:  org.apache.solr.cloud.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete

Error Message:
Error from server at http://127.0.0.1:58006/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 </title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason: <pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/> </body> </html>

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:58006/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/>
</body>
</html>

        at __randomizedtesting.SeedInfo.seed([3A846B1F94E663A1:997EC5BA130E8904]:0)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
        at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
        at org.apache.solr.cloud.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete(TestCollectionsAPIViaSolrCloudCluster.java:167)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58006/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/>
</body>
</html>

        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:591)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:516)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        ... 1 more


FAILED:  org.apache.solr.update.TestInPlaceUpdatesDistrib.test

Error Message:
This doc was supposed to have been deleted, but was: SolrDocument{id=1, inplace_updatable_float=1.0, _version_=1577636011774574592, inplace_updatable_int_with_default=666, inplace_updatable_float_with_default=42.0}

Stack Trace:
java.lang.AssertionError: This doc was supposed to have been deleted, but was: SolrDocument{id=1, inplace_updatable_float=1.0, _version_=1577636011774574592, inplace_updatable_int_with_default=666, inplace_updatable_float_with_default=42.0}
        at __randomizedtesting.SeedInfo.seed([3A846B1F94E663A1:B2D054C53A1A0E59]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at org.apache.solr.update.TestInPlaceUpdatesDistrib.delayedReorderingFetchesMissingUpdateFromLeaderTest(TestInPlaceUpdatesDistrib.java:933)
        at org.apache.solr.update.TestInPlaceUpdatesDistrib.test(TestInPlaceUpdatesDistrib.java:145)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 11213 lines...]
   [junit4] Suite: org.apache.solr.update.TestInPlaceUpdatesDistrib
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/init-core-data-001
   [junit4]   2> 139195 WARN  (SUITE-TestInPlaceUpdatesDistrib-seed#[3A846B1F94E663A1]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=5 numCloses=5
   [junit4]   2> 139195 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[3A846B1F94E663A1]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 139239 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[3A846B1F94E663A1]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 139240 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[3A846B1F94E663A1]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 139241 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[3A846B1F94E663A1]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /zv_jr/
   [junit4]   2> 139242 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[3A846B1F94E663A1]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 139430 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[3A846B1F94E663A1]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 139653 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[3A846B1F94E663A1]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 139934 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[3A846B1F94E663A1]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=inplace-updates
   [junit4]   2> 139938 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[3A846B1F94E663A1]-worker) [    ] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
   [junit4]   2> 140034 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[3A846B1F94E663A1]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 140034 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[3A846B1F94E663A1]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 140034 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[3A846B1F94E663A1]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 140051 INFO  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 140099 INFO  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 140147 INFO  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=inplace-updates
   [junit4]   2> 140151 INFO  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
   [junit4]   2> 140151 INFO  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 140155 INFO  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 140155 INFO  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 140155 INFO  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/init-core-data-001/]
   [junit4]   2> 140207 WARN  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 140999 INFO  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 140999 INFO  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 141002 INFO  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 141002 INFO  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 141004 INFO  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@85abe5[collection1] main]
   [junit4]   2> 141005 WARN  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 141005 INFO  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 141006 INFO  (coreLoadExecutor-191-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577635914087137280
   [junit4]   2> 141008 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[3A846B1F94E663A1]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 141033 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 141051 INFO  (Thread-43) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 141051 INFO  (Thread-43) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 141053 INFO  (searcherExecutor-192-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@85abe5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 141118 ERROR (Thread-43) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 141151 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.ZkTestServer start zk server on port:60939
   [junit4]   2> 141277 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 142441 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/schema-inplace-updates.xml to /configs/conf1/schema.xml
   [junit4]   2> 142442 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 142445 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 142446 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 142446 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 142449 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 142451 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 142452 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 142456 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 142460 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 142535 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 144217 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 144483 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2d8f1ada{/zv_jr,null,AVAILABLE}
   [junit4]   2> 144564 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@420a48f8{SSL,[ssl, http/1.1]}{127.0.0.1:50568}
   [junit4]   2> 144565 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.e.j.s.Server Started @152484ms
   [junit4]   2> 144565 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/tempDir-001/control/data, hostContext=/zv_jr, hostPort=50568, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/control-001/cores}
   [junit4]   2> 144607 ERROR (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 144612 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 144612 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 144612 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 144615 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-04T18:47:35.514Z
   [junit4]   2> 144670 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 144670 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/control-001/solr.xml
   [junit4]   2> 144678 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 144684 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60939/solr
   [junit4]   2> 145032 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 145036 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:50568_zv_jr
   [junit4]   2> 145037 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.Overseer Overseer (id=98602244635557893-127.0.0.1:50568_zv_jr-n_0000000000) starting
   [junit4]   2> 145235 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50568_zv_jr
   [junit4]   2> 145254 INFO  (zkCallback-107-thread-1-processing-n:127.0.0.1:50568_zv_jr) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 145648 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:50568_zv_jr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 145648 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:50568_zv_jr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 145648 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:50568_zv_jr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 145651 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/control-001/cores
   [junit4]   2> 145942 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 145965 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60939/solr ready
   [junit4]   2> 148065 INFO  (qtp962481920-478) [n:127.0.0.1:50568_zv_jr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:50568_zv_jr&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 148120 INFO  (OverseerThreadFactory-202-thread-1-processing-n:127.0.0.1:50568_zv_jr) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 148779 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 148780 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 148889 INFO  (zkCallback-107-thread-1-processing-n:127.0.0.1:50568_zv_jr) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 149933 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 150072 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=inplace-updates
   [junit4]   2> 150086 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
   [junit4]   2> 150086 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 150090 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 150091 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 150091 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 150153 WARN  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 150529 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 150529 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 150530 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 150530 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 150532 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1f72d109[control_collection_shard1_replica_n1] main]
   [junit4]   2> 150535 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 150536 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 150537 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 150550 INFO  (searcherExecutor-205-thread-1-processing-n:127.0.0.1:50568_zv_jr x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@1f72d109[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 150550 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577635924094746624
   [junit4]   2> 150587 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 150587 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 150587 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:50568/zv_jr/control_collection_shard1_replica_n1/
   [junit4]   2> 150588 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 150596 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:50568/zv_jr/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 150596 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 150632 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:50568/zv_jr/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 150733 INFO  (zkCallback-107-thread-1-processing-n:127.0.0.1:50568_zv_jr) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 150783 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 150786 INFO  (qtp962481920-481) [n:127.0.0.1:50568_zv_jr c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2007
   [junit4]   2> 150888 INFO  (zkCallback-107-thread-1-processing-n:127.0.0.1:50568_zv_jr) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 151058 INFO  (qtp962481920-478) [n:127.0.0.1:50568_zv_jr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 151058 INFO  (qtp962481920-478) [n:127.0.0.1:50568_zv_jr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:50568_zv_jr&wt=javabin&version=2} status=0 QTime=3003
   [junit4]   2> 151139 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 151140 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60939/solr ready
   [junit4]   2> 151142 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 151149 INFO  (qtp962481920-479) [n:127.0.0.1:50568_zv_jr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 151180 INFO  (OverseerCollectionConfigSetProcessor-98602244635557893-127.0.0.1:50568_zv_jr-n_0000000000) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 151192 INFO  (OverseerThreadFactory-202-thread-2-processing-n:127.0.0.1:50568_zv_jr) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 151193 WARN  (OverseerThreadFactory-202-thread-2-processing-n:127.0.0.1:50568_zv_jr) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 151535 INFO  (qtp962481920-479) [n:127.0.0.1:50568_zv_jr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 151543 INFO  (qtp962481920-479) [n:127.0.0.1:50568_zv_jr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=393
   [junit4]   2> 151782 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/shard-1-001 of type NRT
   [junit4]   2> 151813 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 151935 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3551c51c{/zv_jr,null,AVAILABLE}
   [junit4]   2> 151936 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@518c5ca4{SSL,[ssl, http/1.1]}{127.0.0.1:60593}
   [junit4]   2> 151936 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.e.j.s.Server Started @159855ms
   [junit4]   2> 151936 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/zv_jr, hostPort=60593, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/shard-1-001/cores}
   [junit4]   2> 151941 ERROR (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 151941 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 151941 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 151941 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 151941 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-04T18:47:42.840Z
   [junit4]   2> 151961 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 151961 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/shard-1-001/solr.xml
   [junit4]   2> 151973 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 151993 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60939/solr
   [junit4]   2> 152150 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:60593_zv_jr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 152153 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:60593_zv_jr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 152156 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:60593_zv_jr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60593_zv_jr
   [junit4]   2> 152159 INFO  (zkCallback-114-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 152159 INFO  (zkCallback-107-thread-1-processing-n:127.0.0.1:50568_zv_jr) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 152183 INFO  (zkCallback-119-thread-1-processing-n:127.0.0.1:60593_zv_jr) [n:127.0.0.1:60593_zv_jr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 152723 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:60593_zv_jr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 152766 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:60593_zv_jr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 152766 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:60593_zv_jr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 152768 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:60593_zv_jr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/shard-1-001/cores
   [junit4]   2> 152894 INFO  (qtp1400812606-518) [n:127.0.0.1:60593_zv_jr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:60593_zv_jr&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 152928 INFO  (OverseerCollectionConfigSetProcessor-98602244635557893-127.0.0.1:50568_zv_jr-n_0000000000) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 152945 INFO  (OverseerThreadFactory-202-thread-3-processing-n:127.0.0.1:50568_zv_jr) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:60593_zv_jr for creating new replica
   [junit4]   2> 152964 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 152964 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 153070 INFO  (zkCallback-119-thread-1-processing-n:127.0.0.1:60593_zv_jr) [n:127.0.0.1:60593_zv_jr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 154019 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 154064 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema [collection1_shard1_replica_n21] Schema name=inplace-updates
   [junit4]   2> 154068 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
   [junit4]   2> 154068 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n21' using configuration from collection collection1, trusted=true
   [junit4]   2> 154082 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n21' (registry 'solr.core.collection1.shard1.replica_n21') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 154082 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 154082 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [[collection1_shard1_replica_n21] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/shard-1-001/cores/collection1_shard1_replica_n21], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/shard-1-001/cores/collection1_shard1_replica_n21/data/]
   [junit4]   2> 154146 WARN  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 154641 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 154641 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 154648 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 154648 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 154654 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.s.SolrIndexSearcher Opening [Searcher@20580d5e[collection1_shard1_replica_n21] main]
   [junit4]   2> 154657 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 154658 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 154659 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 154660 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577635928404393984
   [junit4]   2> 154665 INFO  (searcherExecutor-216-thread-1-processing-n:127.0.0.1:60593_zv_jr x:collection1_shard1_replica_n21 s:shard1 c:collection1) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [collection1_shard1_replica_n21] Registered new searcher Searcher@20580d5e[collection1_shard1_replica_n21] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 154680 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 154680 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 154680 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:60593/zv_jr/collection1_shard1_replica_n21/
   [junit4]   2> 154680 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 154680 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy https://127.0.0.1:60593/zv_jr/collection1_shard1_replica_n21/ has no replicas
   [junit4]   2> 154681 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 154686 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:60593/zv_jr/collection1_shard1_replica_n21/ shard1
   [junit4]   2> 154789 INFO  (zkCallback-119-thread-1-processing-n:127.0.0.1:60593_zv_jr) [n:127.0.0.1:60593_zv_jr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 155021 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 155027 INFO  (qtp1400812606-520) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n21] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2063
   [junit4]   2> 155037 INFO  (qtp1400812606-518) [n:127.0.0.1:60593_zv_jr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:60593_zv_jr&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=2142
   [junit4]   2> 155049 INFO  (OverseerCollectionConfigSetProcessor-98602244635557893-127.0.0.1:50568_zv_jr-n_0000000000) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 155135 INFO  (zkCallback-119-thread-1-processing-n:127.0.0.1:60593_zv_jr) [n:127.0.0.1:60593_zv_jr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 155462 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/shard-2-001 of type NRT
   [junit4]   2> 155528 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 155648 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@571e7238{/zv_jr,null,AVAILABLE}
   [junit4]   2> 155649 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@6b566ad4{SSL,[ssl, http/1.1]}{127.0.0.1:43436}
   [junit4]   2> 155649 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.e.j.s.Server Started @163568ms
   [junit4]   2> 155649 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/zv_jr, hostPort=43436, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/shard-2-001/cores}
   [junit4]   2> 155649 ERROR (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 155650 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 155650 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 155650 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 155650 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-04T18:47:46.549Z
   [junit4]   2> 155681 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 155681 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/shard-2-001/solr.xml
   [junit4]   2> 155691 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 155717 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60939/solr
   [junit4]   2> 155774 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:43436_zv_jr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 155776 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:43436_zv_jr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 155778 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:43436_zv_jr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43436_zv_jr
   [junit4]   2> 155780 INFO  (zkCallback-119-thread-1-processing-n:127.0.0.1:60593_zv_jr) [n:127.0.0.1:60593_zv_jr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 155814 INFO  (zkCallback-125-thread-1-processing-n:127.0.0.1:43436_zv_jr) [n:127.0.0.1:43436_zv_jr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 155783 INFO  (zkCallback-114-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 155780 INFO  (zkCallback-107-thread-1-processing-n:127.0.0.1:50568_zv_jr) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 156040 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:43436_zv_jr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 156125 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:43436_zv_jr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 156126 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:43436_zv_jr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 156149 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:43436_zv_jr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/shard-2-001/cores
   [junit4]   2> 156501 INFO  (qtp1400812606-519) [n:127.0.0.1:60593_zv_jr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:43436_zv_jr&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 156510 INFO  (OverseerThreadFactory-202-thread-4-processing-n:127.0.0.1:50568_zv_jr) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:43436_zv_jr for creating new replica
   [junit4]   2> 156585 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n23&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 156585 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 156690 INFO  (zkCallback-119-thread-1-processing-n:127.0.0.1:60593_zv_jr) [n:127.0.0.1:60593_zv_jr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 156690 INFO  (zkCallback-125-thread-1-processing-n:127.0.0.1:43436_zv_jr) [n:127.0.0.1:43436_zv_jr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 157605 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 157651 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.s.IndexSchema [collection1_shard1_replica_n23] Schema name=inplace-updates
   [junit4]   2> 157663 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
   [junit4]   2> 157663 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n23' using configuration from collection collection1, trusted=true
   [junit4]   2> 157663 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n23' (registry 'solr.core.collection1.shard1.replica_n23') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 157664 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 157664 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.c.SolrCore [[collection1_shard1_replica_n23] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/shard-2-001/cores/collection1_shard1_replica_n23], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/shard-2-001/cores/collection1_shard1_replica_n23/data/]
   [junit4]   2> 157692 WARN  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 157860 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 157860 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 157861 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 157861 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 157865 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.s.SolrIndexSearcher Opening [Searcher@4164503[collection1_shard1_replica_n23] main]
   [junit4]   2> 157866 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 157866 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 157867 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 157868 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577635931768225792
   [junit4]   2> 157873 INFO  (searcherExecutor-227-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.c.SolrCore [collection1_shard1_replica_n23] Registered new searcher Searcher@4164503[collection1_shard1_replica_n23] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 157878 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.c.ZkController Core needs to recover:collection1_shard1_replica_n23
   [junit4]   2> 157894 INFO  (qtp347748224-549) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1  x:collection1_shard1_replica_n23] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n23&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1310
   [junit4]   2> 157898 INFO  (qtp1400812606-519) [n:127.0.0.1:60593_zv_jr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:43436_zv_jr&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1397
   [junit4]   2> 157956 INFO  (updateExecutor-122-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 158008 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 158008 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 158009 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard1_replica_n23]
   [junit4]   2> 158009 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 158009 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard1_replica_n23] as recovering, leader is [https://127.0.0.1:60593/zv_jr/collection1_shard1_replica_n21/] and I am [https://127.0.0.1:43436/zv_jr/collection1_shard1_replica_n23/]
   [junit4]   2> 158080 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Sending prep recovery command to [https://127.0.0.1:60593/zv_jr]; [WaitForState: action=PREPRECOVERY&core=collection1_shard1_replica_n21&nodeName=127.0.0.1:43436_zv_jr&coreNodeName=core_node24&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 158091 INFO  (qtp1400812606-521) [n:127.0.0.1:60593_zv_jr    ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node24, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
   [junit4]   2> 158092 INFO  (qtp1400812606-521) [n:127.0.0.1:60593_zv_jr    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n21, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:43436_zv_jr, coreNodeName=core_node24, onlyIfActiveCheckResult=false, nodeProps: core_node24:{"core":"collection1_shard1_replica_n23","base_url":"https://127.0.0.1:43436/zv_jr","node_name":"127.0.0.1:43436_zv_jr","state":"down","type":"NRT"}
   [junit4]   2> 158137 INFO  (zkCallback-119-thread-1-processing-n:127.0.0.1:60593_zv_jr) [n:127.0.0.1:60593_zv_jr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 158137 INFO  (zkCallback-125-thread-1-processing-n:127.0.0.1:43436_zv_jr) [n:127.0.0.1:43436_zv_jr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 158500 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/shard-3-001 of type NRT
   [junit4]   2> 158510 INFO  (OverseerCollectionConfigSetProcessor-98602244635557893-127.0.0.1:50568_zv_jr-n_0000000000) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 158511 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 158568 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@52111727{/zv_jr,null,AVAILABLE}
   [junit4]   2> 158569 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@faf21d1{SSL,[ssl, http/1.1]}{127.0.0.1:46766}
   [junit4]   2> 158569 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.e.j.s.Server Started @166488ms
   [junit4]   2> 158569 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/zv_jr, hostPort=46766, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/shard-3-001/cores}
   [junit4]   2> 158569 ERROR (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 158576 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 158576 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 158576 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 158576 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-04T18:47:49.475Z
   [junit4]   2> 158610 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 158610 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/shard-3-001/solr.xml
   [junit4]   2> 158618 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 158634 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60939/solr
   [junit4]   2> 158682 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:46766_zv_jr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 158685 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:46766_zv_jr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 158691 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:46766_zv_jr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46766_zv_jr
   [junit4]   2> 158693 INFO  (zkCallback-114-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 158704 INFO  (zkCallback-107-thread-2-processing-n:127.0.0.1:50568_zv_jr) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 158711 INFO  (zkCallback-125-thread-1-processing-n:127.0.0.1:43436_zv_jr) [n:127.0.0.1:43436_zv_jr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 158710 INFO  (zkCallback-119-thread-1-processing-n:127.0.0.1:60593_zv_jr) [n:127.0.0.1:60593_zv_jr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 158730 INFO  (zkCallback-132-thread-1-processing-n:127.0.0.1:46766_zv_jr) [n:127.0.0.1:46766_zv_jr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 159092 INFO  (qtp1400812606-521) [n:127.0.0.1:60593_zv_jr    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n21, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:43436_zv_jr, coreNodeName=core_node24, onlyIfActiveCheckResult=false, nodeProps: core_node24:{"core":"collection1_shard1_replica_n23","base_url":"https://127.0.0.1:43436/zv_jr","node_name":"127.0.0.1:43436_zv_jr","state":"recovering","type":"NRT"}
   [junit4]   2> 159092 INFO  (qtp1400812606-521) [n:127.0.0.1:60593_zv_jr    ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node24, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
   [junit4]   2> 159092 INFO  (qtp1400812606-521) [n:127.0.0.1:60593_zv_jr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:43436_zv_jr&onlyIfLeaderActive=true&core=collection1_shard1_replica_n21&coreNodeName=core_node24&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1001
   [junit4]   2> 159617 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [https://127.0.0.1:60593/zv_jr/collection1_shard1_replica_n21/] - recoveringAfterStartup=[true]
   [junit4]   2> 159626 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica_n23 url=https://127.0.0.1:43436/zv_jr START replicas=[https://127.0.0.1:60593/zv_jr/collection1_shard1_replica_n21/] nUpdates=100
   [junit4]   2> 159761 INFO  (qtp1400812606-521) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1 r:core_node22 x:collection1_shard1_replica_n21] o.a.s.u.IndexFingerprint IndexFingerprint millis:8.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
   [junit4]   2> 159761 INFO  (qtp1400812606-521) [n:127.0.0.1:60593_zv_jr c:collection1 s:shard1 r:core_node22 x:collection1_shard1_replica_n21] o.a.s.c.S.Request [collection1_shard1_replica_n21]  webapp=/zv_jr path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=12
   [junit4]   2> 159806 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.IndexFingerprint IndexFingerprint millis:43.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
   [junit4]   2> 159823 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
   [junit4]   2> 159824 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 159824 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 159824 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 159824 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
   [junit4]   2> 159824 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
   [junit4]   2> 159824 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy No replay needed.
   [junit4]   2> 159824 INFO  (recoveryExecutor-123-thread-1-processing-n:127.0.0.1:43436_zv_jr x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:43436_zv_jr c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
   [junit4]   2> 159930 INFO  (zkCallback-119-thread-1-processing-n:127.0.0.1:60593_zv_jr) [n:127.0.0.1:60593_zv_jr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 159930 INFO  (zkCallback-125-thread-1-processing-n:127.0.0.1:43436_zv_jr) [n:127.0.0.1:43436_zv_jr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 160175 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:46766_zv_jr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 160304 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:46766_zv_jr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 160306 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:46766_zv_jr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42d830ee
   [junit4]   2> 160308 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[3A846B1F94E663A1]) [n:127.0.0.1:46766_zv_jr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_3A846B1F94E663A1-001/shard-3-001/cores
   [junit4]   2> 161098 INFO  (qtp1073200716-585) [n:127.0.0.1:46766_zv_jr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:46766_zv_jr&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 161114 INFO  (OverseerThreadFactory-202-thread-5-processing-n:127.0.0.1:50568_zv_jr) [n:127.0.0.1:50568_zv_jr    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:46766_zv_jr for creating new replica
   [junit4]   2> 161128 INFO  (qtp1073200716-582) [n:127.0.0.1:46766_zv_jr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n25&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 161128 INFO  (qtp1073200716-582) [n:127.0.0.1:46766_zv_jr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 161233 INFO  (zkCallback-125-thread-1-processing-n:127.0.0.1:43436_zv_jr) [n:127.0.0.1:43436_zv_jr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 161233 INFO  (zkCallback-119-thread-1-processing-n:127.0.0.1:60593_zv_jr) [n:127.0.0.1:60593_zv_jr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConne

[...truncated too long message...]

extHandler Stopped o.e.j.s.ServletContextHandler@5d2b910b{/solr,null,UNAVAILABLE}
   [junit4]   2> 1304251 WARN  (zkCallback-2497-thread-2-processing-n:127.0.0.1:50466_solr) [n:127.0.0.1:50466_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1304252 INFO  (jetty-closer-2451-thread-5) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4ca4ca8c{/solr,null,UNAVAILABLE}
   [junit4]   2> 1304298 WARN  (zkCallback-2472-thread-3-processing-n:127.0.0.1:51241_solr) [n:127.0.0.1:51241_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1304298 INFO  (jetty-closer-2451-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1f67ae96{/solr,null,UNAVAILABLE}
   [junit4]   2> 1304299 ERROR (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3A846B1F94E663A1]) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1304300 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3A846B1F94E663A1]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37091 37091
   [junit4]   2> 1304425 INFO  (Thread-1300) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37091 37091
   [junit4]   2> 1304426 WARN  (Thread-1300) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 8 /solr/aliases.json
   [junit4]   2> 8 /solr/clusterprops.json
   [junit4]   2> 8 /solr/configs/solrCloudCollectionConfig
   [junit4]   2> 7 /solr/security.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 61 /solr/collections/testcollection/state.json
   [junit4]   2> 8 /solr/clusterstate.json
   [junit4]   2> 3 /solr/overseer_elect/election/98602319657304075-127.0.0.1:51241_solr-n_0000000000
   [junit4]   2> 2 /solr/overseer_elect/election/98602319657304079-127.0.0.1:58006_solr-n_0000000002
   [junit4]   2> 2 /solr/overseer_elect/election/98602319657304076-127.0.0.1:53591_solr-n_0000000004
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 8 /solr/collections
   [junit4]   2> 6 /solr/live_nodes
   [junit4]   2>
   [junit4]   2> 1304427 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3A846B1F94E663A1]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testCollectionCreateSearchDelete
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestCollectionsAPIViaSolrCloudCluster -Dtests.method=testCollectionCreateSearchDelete -Dtests.seed=3A846B1F94E663A1 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=sr-Latn-BA -Dtests.timezone=America/Antigua -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   18.6s J1 | TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:58006/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
   [junit4]    > <head>
   [junit4]    > <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
   [junit4]    > <title>Error 404 </title>
   [junit4]    > </head>
   [junit4]    > <body>
   [junit4]    > <h2>HTTP ERROR: 404</h2>
   [junit4]    > <p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
   [junit4]    > <pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p>
   [junit4]    > <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/>
   [junit4]    > </body>
   [junit4]    > </html>
   [junit4]    > at __randomizedtesting.SeedInfo.seed([3A846B1F94E663A1:997EC5BA130E8904]:0)
   [junit4]    > at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539)
   [junit4]    > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993)
   [junit4]    > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
   [junit4]    > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
   [junit4]    > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
   [junit4]    > at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
   [junit4]    > at org.apache.solr.cloud.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete(TestCollectionsAPIViaSolrCloudCluster.java:167)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58006/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
   [junit4]    > <head>
   [junit4]    > <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
   [junit4]    > <title>Error 404 </title>
   [junit4]    > </head>
   [junit4]    > <body>
   [junit4]    > <h2>HTTP ERROR: 404</h2>
   [junit4]    > <p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
   [junit4]    > <pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p>
   [junit4]    > <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/>
   [junit4]    > </body>
   [junit4]    > </html>
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:591)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
   [junit4]    > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
   [junit4]    > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
   [junit4]    > at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:516)
   [junit4]    > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]    > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]    > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]    > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    > ... 1 more
   [junit4]   2> 1304429 INFO  (SUITE-TestCollectionsAPIViaSolrCloudCluster-seed#[3A846B1F94E663A1]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37091 37091
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3A846B1F94E663A1-001
   [junit4]   2> Sep 04, 2017 7:06:55 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true): {}, locale=sr-Latn-BA, timezone=America/Antigua
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=158022832,total=514850816
   [junit4]   2> NOTE: All tests run in this JVM: [TestFieldCacheReopen, PrimUtilsTest, SchemaVersionSpecificBehaviorTest, MoreLikeThisHandlerTest, FileBasedSpellCheckerTest, GraphQueryTest, ReplicaListTransformerTest, SubstringBytesRefFilterTest, OverseerTaskQueueTest, ZkCLITest, TestHashPartitioner, TestEmbeddedSolrServerConstructors, BadComponentTest, DistanceFunctionTest, AnalysisAfterCoreReloadTest, TestInPlaceUpdatesDistrib, SoftAutoCommitTest, SpatialFilterTest, TestSystemIdResolver, StatelessScriptUpdateProcessorFactoryTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, ForceLeaderTest, TestFreeTextSuggestions, TestReplicationHandlerBackup, TestSolrCoreSnapshots, ZkFailoverTest, DocValuesTest, HdfsRecoverLeaseTest, ResourceLoaderTest, TestQueryTypes, AnalyticsMergeStrategyTest, UpdateRequestProcessorFactoryTest, SaslZkACLProviderTest, TestDocBasedVersionConstraints, BlobRepositoryCloudTest, SolrMetricReporterTest, ClusterStateUpdateTest, TestPostingsSolrHighlighter, ZkSolrClientTest, HdfsChaosMonkeySafeLeaderTest, TestGraphMLResponseWriter, JsonLoaderTest, TestLegacyFieldReuse, TestConfigSetProperties, TestCursorMarkWithoutUniqueKey, DeleteInactiveReplicaTest, TestSolrFieldCacheBean, IndexSchemaTest, HLLSerializationTest, ClassificationUpdateProcessorTest, EnumFieldTest, ShardRoutingTest, VersionInfoTest, TestPullReplica, OpenCloseCoreStressTest, DistributedQueueTest, TestComplexPhraseLeadingWildcard, SuggestComponentContextFilterQueryTest, LoggingHandlerTest, TestLRUCache, QueryParsingTest, TestSchemalessBufferedUpdates, TestSearchPerf, ImplicitSnitchTest, BitVectorTest, TestSQLHandler, TestPartialUpdateDeduplication, BasicFunctionalityTest, TestScoreJoinQPNoScore, TestInfoStreamLogging, TestManagedStopFilterFactory, TestBlobHandler, ConnectionManagerTest, TestConfigsApi, MoveReplicaTest, HdfsUnloadDistributedZkTest, DistributedSuggestComponentTest, SolrCloudExampleTest, TestCoreContainer, BadCopyFieldTest, TestJsonFacetRefinement, TolerantUpdateProcessorTest, CleanupOldIndexTest, TestCollectionsAPIViaSolrCloudCluster]
   [junit4] Completed [258/732 (2!)] on J1 in 55.24s, 3 tests, 1 error <<< FAILURES!

[...truncated 47689 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2089 - Failure

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2089/

4 tests failed.
FAILED:  org.apache.solr.cloud.LeaderElectionContextKeyTest.test

Error Message:


Stack Trace:
java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([500F99E268BE63A1:D85BA638C6420E59]:0)
        at org.junit.Assert.fail(Assert.java:92)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertTrue(Assert.java:54)
        at org.apache.solr.cloud.LeaderElectionContextKeyTest.test(LeaderElectionContextKeyTest.java:88)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores:     1) Thread[id=8819, name=searcherExecutor-1721-thread-1, state=WAITING, group=TGRP-TestLazyCores]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores:
   1) Thread[id=8819, name=searcherExecutor-1721-thread-1, state=WAITING, group=TGRP-TestLazyCores]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([500F99E268BE63A1]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=8819, name=searcherExecutor-1721-thread-1, state=WAITING, group=TGRP-TestLazyCores]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=8819, name=searcherExecutor-1721-thread-1, state=WAITING, group=TGRP-TestLazyCores]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([500F99E268BE63A1]:0)


FAILED:  org.apache.solr.core.TestLazyCores.testNoCommit

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
        at __randomizedtesting.SeedInfo.seed([500F99E268BE63A1:8F6F3833A3990004]:0)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:884)
        at org.apache.solr.core.TestLazyCores.check10(TestLazyCores.java:847)
        at org.apache.solr.core.TestLazyCores.testNoCommit(TestLazyCores.java:829)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound='10']
        xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">1</int><lst name="params"><str name="q">*:*</str></lst></lst><result name="response" numFound="0" start="0"></result>
</response>

        request was:q=*:*
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:877)
        ... 41 more




Build Log:
[...truncated 11444 lines...]
   [junit4] Suite: org.apache.solr.core.TestLazyCores
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/init-core-data-001
   [junit4]   2> 627138 WARN  (SUITE-TestLazyCores-seed#[500F99E268BE63A1]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=24 numCloses=24
   [junit4]   2> 627138 INFO  (SUITE-TestLazyCores-seed#[500F99E268BE63A1]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 627139 INFO  (SUITE-TestLazyCores-seed#[500F99E268BE63A1]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 627140 INFO  (SUITE-TestLazyCores-seed#[500F99E268BE63A1]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 627141 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCreateTransientFromAdmin
   [junit4]   2> 627151 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/solr.xml
   [junit4]   2> 627167 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 627229 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 627248 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 627248 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 627259 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 627299 INFO  (coreLoadExecutor-1618-thread-2) [    x:collection2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 627310 INFO  (coreLoadExecutor-1618-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 627302 INFO  (coreLoadExecutor-1618-thread-3) [    x:collection5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 627324 INFO  (coreLoadExecutor-1618-thread-3) [    x:collection5] o.a.s.s.IndexSchema [collection5] Schema name=tiny
   [junit4]   2> 627333 INFO  (coreLoadExecutor-1618-thread-2) [    x:collection2] o.a.s.s.IndexSchema [collection2] Schema name=tiny
   [junit4]   2> 627336 INFO  (coreLoadExecutor-1618-thread-2) [    x:collection2] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 627336 INFO  (coreLoadExecutor-1618-thread-2) [    x:collection2] o.a.s.c.CoreContainer Creating SolrCore 'collection2' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./collection2, trusted=true
   [junit4]   2> 627336 INFO  (coreLoadExecutor-1618-thread-2) [    x:collection2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection2' (registry 'solr.core.collection2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 627336 INFO  (coreLoadExecutor-1618-thread-2) [    x:collection2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 627337 INFO  (coreLoadExecutor-1618-thread-2) [    x:collection2] o.a.s.c.SolrCore [[collection2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/collection2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./collection2/data/]
   [junit4]   2> 627339 INFO  (coreLoadExecutor-1618-thread-2) [    x:collection2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=20, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 627341 INFO  (coreLoadExecutor-1618-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=tiny
   [junit4]   2> 627343 INFO  (coreLoadExecutor-1618-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 627343 INFO  (coreLoadExecutor-1618-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./collection1, trusted=true
   [junit4]   2> 627343 INFO  (coreLoadExecutor-1618-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 627343 INFO  (coreLoadExecutor-1618-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 627343 INFO  (coreLoadExecutor-1618-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./collection1/data/]
   [junit4]   2> 627356 INFO  (coreLoadExecutor-1618-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=20, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 627374 INFO  (coreLoadExecutor-1618-thread-3) [    x:collection5] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 627375 INFO  (coreLoadExecutor-1618-thread-3) [    x:collection5] o.a.s.c.CoreContainer Creating SolrCore 'collection5' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./collection5, trusted=true
   [junit4]   2> 627375 INFO  (coreLoadExecutor-1618-thread-3) [    x:collection5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection5' (registry 'solr.core.collection5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 627375 INFO  (coreLoadExecutor-1618-thread-3) [    x:collection5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 627375 INFO  (coreLoadExecutor-1618-thread-3) [    x:collection5] o.a.s.c.SolrCore [[collection5] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/collection5], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./collection5/data/]
   [junit4]   2> 627378 INFO  (coreLoadExecutor-1618-thread-3) [    x:collection5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=20, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 627468 INFO  (coreLoadExecutor-1618-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 627468 INFO  (coreLoadExecutor-1618-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 627469 INFO  (coreLoadExecutor-1618-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=318168228, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 627471 INFO  (coreLoadExecutor-1618-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4f831cb8[collection1] main]
   [junit4]   2> 627471 INFO  (coreLoadExecutor-1618-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/collection1/conf
   [junit4]   2> 627472 INFO  (coreLoadExecutor-1618-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 627472 INFO  (coreLoadExecutor-1618-thread-3) [    x:collection5] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 627472 INFO  (coreLoadExecutor-1618-thread-3) [    x:collection5] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 627479 INFO  (coreLoadExecutor-1618-thread-2) [    x:collection2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 627479 INFO  (coreLoadExecutor-1618-thread-2) [    x:collection2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 627480 INFO  (searcherExecutor-1620-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4f831cb8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 627480 INFO  (coreLoadExecutor-1618-thread-2) [    x:collection2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=318168228, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 627480 INFO  (coreLoadExecutor-1618-thread-3) [    x:collection5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=318168228, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 627481 INFO  (coreLoadExecutor-1618-thread-2) [    x:collection2] o.a.s.s.SolrIndexSearcher Opening [Searcher@6afe1328[collection2] main]
   [junit4]   2> 627481 INFO  (coreLoadExecutor-1618-thread-2) [    x:collection2] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/collection2/conf
   [junit4]   2> 627482 INFO  (coreLoadExecutor-1618-thread-2) [    x:collection2] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 627482 INFO  (coreLoadExecutor-1618-thread-3) [    x:collection5] o.a.s.s.SolrIndexSearcher Opening [Searcher@6a915963[collection5] main]
   [junit4]   2> 627482 INFO  (coreLoadExecutor-1618-thread-3) [    x:collection5] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/collection5/conf
   [junit4]   2> 627483 INFO  (coreLoadExecutor-1618-thread-3) [    x:collection5] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 627489 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    ] o.a.s.h.a.CoreAdminOperation core create command loadOnStartup=true&transient=true&name=core1&action=CREATE&wt=xml
   [junit4]   2> 627489 INFO  (searcherExecutor-1621-thread-1-processing-x:collection5) [    x:collection5] o.a.s.c.SolrCore [collection5] Registered new searcher Searcher@6a915963[collection5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 627514 INFO  (searcherExecutor-1619-thread-1-processing-x:collection2) [    x:collection2] o.a.s.c.SolrCore [collection2] Registered new searcher Searcher@6afe1328[collection2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 627515 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 627522 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema [core1] Schema name=tiny
   [junit4]   2> 627537 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 627538 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core1, trusted=true
   [junit4]   2> 627538 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.core1' (registry 'solr.core.core1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 627538 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 627538 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore [[core1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core1/data/]
   [junit4]   2> 627541 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=21.259765625, floorSegmentMB=0.861328125, forceMergeDeletesPctAllowed=23.133015769239655, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 627641 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 627641 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 627643 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=48, maxMergedSegmentMB=12.6376953125, floorSegmentMB=2.0341796875, forceMergeDeletesPctAllowed=0.7017382034380537, segmentsPerTier=49.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 627643 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@14a5b5b6[core1] main]
   [junit4]   2> 627644 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core1/conf
   [junit4]   2> 627644 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 627645 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.h.a.CoreAdminOperation core create command loadOnStartup=false&transient=true&name=core2&action=CREATE&wt=xml
   [junit4]   2> 627809 INFO  (searcherExecutor-1632-thread-1-processing-x:core1) [    x:core1] o.a.s.c.SolrCore [core1] Registered new searcher Searcher@14a5b5b6[core1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 627811 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 627820 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema [core2] Schema name=tiny
   [junit4]   2> 627823 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 627823 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core2' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core2, trusted=true
   [junit4]   2> 627823 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.core2' (registry 'solr.core.core2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 627824 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 627824 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore [[core2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core2/data/]
   [junit4]   2> 627826 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=33, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 627939 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 627939 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 627940 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=64.171875, floorSegmentMB=2.0166015625, forceMergeDeletesPctAllowed=15.530217359065354, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 627941 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@30c590fb[core2] main]
   [junit4]   2> 627941 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core2/conf
   [junit4]   2> 627942 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 627942 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.h.a.CoreAdminOperation core create command loadOnStartup=true&transient=true&name=core3&action=CREATE&wt=xml
   [junit4]   2> 627951 INFO  (searcherExecutor-1637-thread-1-processing-x:core1) [    x:core1] o.a.s.c.SolrCore [core2] Registered new searcher Searcher@30c590fb[core2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 627953 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 627960 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema [core3] Schema name=tiny
   [junit4]   2> 627962 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 627962 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core3' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core3, trusted=true
   [junit4]   2> 627962 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.core3' (registry 'solr.core.core3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 627962 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 627963 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore [[core3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core3], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core3/data/]
   [junit4]   2> 627965 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=69.826171875, floorSegmentMB=0.802734375, forceMergeDeletesPctAllowed=3.2701486156411996, segmentsPerTier=49.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 628044 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 628044 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 628045 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=48, maxMergedSegmentMB=1.4013671875, floorSegmentMB=1.775390625, forceMergeDeletesPctAllowed=23.604075715601724, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.44065628144497093
   [junit4]   2> 628045 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@62b1a7e4[core3] main]
   [junit4]   2> 628046 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core3/conf
   [junit4]   2> 628046 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 628047 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.h.a.CoreAdminOperation core create command loadOnStartup=false&transient=true&name=core4&action=CREATE&wt=xml
   [junit4]   2> 628055 INFO  (searcherExecutor-1642-thread-1-processing-x:core1) [    x:core1] o.a.s.c.SolrCore [core3] Registered new searcher Searcher@62b1a7e4[core3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 628068 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 628075 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema [core4] Schema name=tiny
   [junit4]   2> 628169 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 628169 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core4' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core4, trusted=true
   [junit4]   2> 628170 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.core4' (registry 'solr.core.core4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 628170 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 628170 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore [[core4] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core4], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core4/data/]
   [junit4]   2> 628174 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=95.013671875, floorSegmentMB=1.9970703125, forceMergeDeletesPctAllowed=4.344275688401718, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.11700431334316788
   [junit4]   2> 628367 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 628368 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 628369 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=50, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 628370 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@13db7c63[core4] main]
   [junit4]   2> 628370 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core4/conf
   [junit4]   2> 628371 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 628372 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.TransientSolrCoreCacheDefault Closing transient core [collection2]
   [junit4]   2> 628372 INFO  (Thread-908) [    ] o.a.s.c.SolrCore [collection2]  CLOSING SolrCore org.apache.solr.core.SolrCore@546463f0
   [junit4]   2> 628372 INFO  (Thread-908) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection2, tag=1415865328
   [junit4]   2> 628372 INFO  (Thread-908) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1ecc7dd7: rootName = null, domain = solr.core.collection2, service url = null, agent id = null] for registry solr.core.collection2 / com.codahale.metrics.MetricRegistry@4f8e86cb
   [junit4]   2> 628382 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.h.a.CoreAdminOperation core create command loadOnStartup=false&transient=true&name=core5&action=CREATE&wt=xml
   [junit4]   2> 628386 INFO  (searcherExecutor-1647-thread-1-processing-x:core1) [    x:core1] o.a.s.c.SolrCore [core4] Registered new searcher Searcher@13db7c63[core4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 628396 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 628405 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema [core5] Schema name=tiny
   [junit4]   2> 628409 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 628409 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core5' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core5, trusted=true
   [junit4]   2> 628410 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.core5' (registry 'solr.core.core5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 628410 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 628410 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore [[core5] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core5], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core5/data/]
   [junit4]   2> 628414 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=43, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 628501 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 628501 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 628503 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=46, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 628503 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@62f76106[core5] main]
   [junit4]   2> 628504 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core5/conf
   [junit4]   2> 628505 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 628505 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.TransientSolrCoreCacheDefault Closing transient core [core1]
   [junit4]   2> 628506 INFO  (Thread-908) [    ] o.a.s.c.SolrCore [core1]  CLOSING SolrCore org.apache.solr.core.SolrCore@5afa8594
   [junit4]   2> 628506 INFO  (Thread-908) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.core1, tag=1526367636
   [junit4]   2> 628506 INFO  (Thread-908) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@57755a68: rootName = null, domain = solr.core.core1, service url = null, agent id = null] for registry solr.core.core1 / com.codahale.metrics.MetricRegistry@4576edbb
   [junit4]   2> 628507 INFO  (searcherExecutor-1652-thread-1-processing-x:core1) [    x:core1] o.a.s.c.SolrCore [core5] Registered new searcher Searcher@62f76106[core5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 628536 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 628545 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema [core1] Schema name=tiny
   [junit4]   2> 628548 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 628548 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core1, trusted=true
   [junit4]   2> 628577 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.core1' (registry 'solr.core.core1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 628577 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 628577 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore [[core1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core1/data/]
   [junit4]   2> 628674 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 628674 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 628675 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=37, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 628676 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@29efec4c[core1] main]
   [junit4]   2> 628676 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core1/conf
   [junit4]   2> 628676 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 628677 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.TransientSolrCoreCacheDefault Closing transient core [core2]
   [junit4]   2> 628677 INFO  (Thread-908) [    ] o.a.s.c.SolrCore [core2]  CLOSING SolrCore org.apache.solr.core.SolrCore@18cde3d6
   [junit4]   2> 628677 INFO  (Thread-908) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.core2, tag=416146390
   [junit4]   2> 628677 INFO  (Thread-908) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@25566e05: rootName = null, domain = solr.core.core2, service url = null, agent id = null] for registry solr.core.core2 / com.codahale.metrics.MetricRegistry@3bf70d6a
   [junit4]   2> 628696 INFO  (searcherExecutor-1656-thread-1-processing-x:core1) [    x:core1] o.a.s.c.SolrCore [core1] Registered new searcher Searcher@29efec4c[core1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 628704 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 628711 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema [core2] Schema name=tiny
   [junit4]   2> 628713 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 628713 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core2' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core2, trusted=true
   [junit4]   2> 628730 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.core2' (registry 'solr.core.core2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 628731 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 628731 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore [[core2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core2/data/]
   [junit4]   2> 628767 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 628767 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 628768 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=20, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=75.193359375, floorSegmentMB=0.6259765625, forceMergeDeletesPctAllowed=14.547250814390507, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7897182460430828
   [junit4]   2> 628769 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@540887f[core2] main]
   [junit4]   2> 628769 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core2/conf
   [junit4]   2> 628770 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 628770 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.TransientSolrCoreCacheDefault Closing transient core [core3]
   [junit4]   2> 628770 INFO  (Thread-908) [    ] o.a.s.c.SolrCore [core3]  CLOSING SolrCore org.apache.solr.core.SolrCore@27c94a27
   [junit4]   2> 628770 INFO  (Thread-908) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.core3, tag=667503143
   [junit4]   2> 628770 INFO  (Thread-908) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7d72da1d: rootName = null, domain = solr.core.core3, service url = null, agent id = null] for registry solr.core.core3 / com.codahale.metrics.MetricRegistry@43d7d0c6
   [junit4]   2> 628772 INFO  (searcherExecutor-1660-thread-1-processing-x:core1) [    x:core1] o.a.s.c.SolrCore [core2] Registered new searcher Searcher@540887f[core2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 628789 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 628820 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema [core3] Schema name=tiny
   [junit4]   2> 628823 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 628823 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core3' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core3, trusted=true
   [junit4]   2> 628839 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.core3' (registry 'solr.core.core3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 628840 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 628840 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore [[core3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core3], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core3/data/]
   [junit4]   2> 628873 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 628873 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 628875 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=34, maxMergedSegmentMB=9.931640625, floorSegmentMB=1.46875, forceMergeDeletesPctAllowed=22.926733186218506, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.13965970865263788
   [junit4]   2> 628875 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@53de4829[core3] main]
   [junit4]   2> 628876 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core3/conf
   [junit4]   2> 628876 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 628876 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.TransientSolrCoreCacheDefault Closing transient core [core4]
   [junit4]   2> 628876 INFO  (Thread-908) [    ] o.a.s.c.SolrCore [core4]  CLOSING SolrCore org.apache.solr.core.SolrCore@681d5ac1
   [junit4]   2> 628877 INFO  (Thread-908) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.core4, tag=1746754241
   [junit4]   2> 628877 INFO  (Thread-908) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@11d3676f: rootName = null, domain = solr.core.core4, service url = null, agent id = null] for registry solr.core.core4 / com.codahale.metrics.MetricRegistry@63c44902
   [junit4]   2> 628890 INFO  (searcherExecutor-1664-thread-1-processing-x:core1) [    x:core1] o.a.s.c.SolrCore [core3] Registered new searcher Searcher@53de4829[core3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 628899 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 628907 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema [core4] Schema name=tiny
   [junit4]   2> 628909 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 628909 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core4' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core4, trusted=true
   [junit4]   2> 628926 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.core4' (registry 'solr.core.core4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 628927 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 628927 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore [[core4] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core4], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core4/data/]
   [junit4]   2> 628960 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 628960 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 628962 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=14, maxMergedSegmentMB=3.1083984375, floorSegmentMB=0.640625, forceMergeDeletesPctAllowed=28.73580127842583, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.46967019868574933
   [junit4]   2> 628962 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5a1e79c6[core4] main]
   [junit4]   2> 628962 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core4/conf
   [junit4]   2> 628963 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 628963 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.TransientSolrCoreCacheDefault Closing transient core [core5]
   [junit4]   2> 628963 INFO  (Thread-908) [    ] o.a.s.c.SolrCore [core5]  CLOSING SolrCore org.apache.solr.core.SolrCore@226f86d9
   [junit4]   2> 628963 INFO  (Thread-908) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.core5, tag=577734361
   [junit4]   2> 628963 INFO  (Thread-908) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@40630c3c: rootName = null, domain = solr.core.core5, service url = null, agent id = null] for registry solr.core.core5 / com.codahale.metrics.MetricRegistry@58d39d8c
   [junit4]   2> 628976 INFO  (searcherExecutor-1668-thread-1-processing-x:core1) [    x:core1] o.a.s.c.SolrCore [core4] Registered new searcher Searcher@5a1e79c6[core4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 628985 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 628992 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema [core5] Schema name=tiny
   [junit4]   2> 628995 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 628995 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core5' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core5, trusted=true
   [junit4]   2> 629012 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.core5' (registry 'solr.core.core5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 629012 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 629012 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore [[core5] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core5], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/./core5/data/]
   [junit4]   2> 629047 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 629047 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 629082 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=48, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6218950855855883]
   [junit4]   2> 629083 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1685aff[core5] main]
   [junit4]   2> 629083 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-001/core5/conf
   [junit4]   2> 629084 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 629084 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.TransientSolrCoreCacheDefault Closing transient core [core1]
   [junit4]   2> 629085 INFO  (searcherExecutor-1672-thread-1-processing-x:core1) [    x:core1] o.a.s.c.SolrCore [core5] Registered new searcher Searcher@1685aff[core5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 629089 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.core2, tag=null
   [junit4]   2> 629089 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6cb53462: rootName = null, domain = solr.core.core2, service url = null, agent id = null] for registry solr.core.core2 / com.codahale.metrics.MetricRegistry@3bf70d6a
   [junit4]   2> 629097 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore Core core2 is not yet closed, waiting 100 ms before checking again.
   [junit4]   2> 629197 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore Core core2 is not yet closed, waiting 100 ms before checking again.
   [junit4]   2> 629296 INFO  (Thread-909) [    x:core1] o.a.s.c.SolrCore [core1]  CLOSING SolrCore org.apache.solr.core.SolrCore@54e1c00
   [junit4]   2> 629296 INFO  (Thread-909) [    x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.core1, tag=89005056
   [junit4]   2> 629296 INFO  (Thread-909) [    x:core1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3058c48d: rootName = null, domain = solr.core.core1, service url = null, agent id = null] for registry solr.core.core1 / com.codahale.metrics.MetricRegistry@4576edbb
   [junit4]   2> 629305 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore Core core2 is not yet closed, waiting 100 ms before checking again.
   [junit4]   2> 629308 INFO  (Thread-909) [    x:core1] o.a.s.c.SolrCore [core2]  CLOSING SolrCore org.apache.solr.core.SolrCore@41a75d01
   [junit4]   2> 629308 INFO  (Thread-909) [    x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.core2, tag=1101487361
   [junit4]   2> 629405 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.core3, tag=null
   [junit4]   2> 629406 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5aaf91e5: rootName = null, domain = solr.core.core3, service url = null, agent id = null] for registry solr.core.core3 / com.codahale.metrics.MetricRegistry@43d7d0c6
   [junit4]   2> 629421 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore [core3]  CLOSING SolrCore org.apache.solr.core.SolrCore@327f4e00
   [junit4]   2> 629421 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.core3, tag=847203840
   [junit4]   2> 629422 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.core4, tag=null
   [junit4]   2> 629422 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@54183786: rootName = null, domain = solr.core.core4, service url = null, agent id = null] for registry solr.core.core4 / com.codahale.metrics.MetricRegistry@63c44902
   [junit4]   2> 629430 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore [core4]  CLOSING SolrCore org.apache.solr.core.SolrCore@f6bb4b1
   [junit4]   2> 629430 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.core4, tag=258716849
   [junit4]   2> 629431 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.core5, tag=null
   [junit4]   2> 629432 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@27a131c4: rootName = null, domain = solr.core.core5, service url = null, agent id = null] for registry solr.core.core5 / com.codahale.metrics.MetricRegistry@58d39d8c
   [junit4]   2> 629440 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrCore [core5]  CLOSING SolrCore org.apache.solr.core.SolrCore@141fc535
   [junit4]   2> 629440 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.core5, tag=337626421
   [junit4]   2> 629441 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1863994956
   [junit4]   2> 629441 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 629441 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7ddf5a78: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@55f8bce2
   [junit4]   2> 629445 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 629445 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@32a5b8a3: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@3ce3ae15
   [junit4]   2> 629448 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 629448 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@66c97c55: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@7c010b22
   [junit4]   2> 629455 INFO  (coreCloseExecutor-1681-thread-1-processing-x:core1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@195900ef
   [junit4]   2> 629455 INFO  (coreCloseExecutor-1681-thread-1-processing-x:core1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=425263343
   [junit4]   2> 629455 INFO  (coreCloseExecutor-1681-thread-1-processing-x:core1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@146f78da: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@478a464
   [junit4]   2> 629467 INFO  (coreCloseExecutor-1681-thread-2-processing-x:core1) [    x:collection5] o.a.s.c.SolrCore [collection5]  CLOSING SolrCore org.apache.solr.core.SolrCore@efaf60b
   [junit4]   2> 629468 INFO  (coreCloseExecutor-1681-thread-2-processing-x:core1) [    x:collection5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection5, tag=251328011
   [junit4]   2> 629470 INFO  (coreCloseExecutor-1681-thread-2-processing-x:core1) [    x:collection5] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@43ddce6: rootName = null, domain = solr.core.collection5, service url = null, agent id = null] for registry solr.core.collection5 / com.codahale.metrics.MetricRegistry@56a5a836
   [junit4]   2> 629485 INFO  (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.SolrTestCaseJ4 ###Ending testCreateTransientFromAdmin
   [junit4]   2> 629493 INFO  (TEST-TestLazyCores.testNoCommit-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.SolrTestCaseJ4 ###Starting testNoCommit
   [junit4]   2> 629502 INFO  (TEST-TestLazyCores.testNoCommit-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-002/solr.xml
   [junit4]   2> 629507 INFO  (TEST-TestLazyCores.testNoCommit-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 629534 INFO  (TEST-TestLazyCores.testNoCommit-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 629542 INFO  (TEST-TestLazyCores.testNoCommit-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 629542 INFO  (TEST-TestLazyCores.testNoCommit-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 629544 INFO  (TEST-TestLazyCores.testNoCommit-seed#[500F99E268BE63A1]) [    x:core1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 629560 INFO  (coreLoadExecutor-1688-thread-1-processing-x:core1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 629575 INFO  (coreLoadExecutor-1688-thread-2-processing-x:core1) [    x:collection2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 629582 INFO  (coreLoadExecutor-1688-thread-1-processing-x:core1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=tiny
   [junit4]   2> 629589 INFO  (coreLoadExecutor-1688-thread-2-processing-x:core1) [    x:collection2] o.a.s.s.IndexSchema [collection2] Schema name=tiny
   [junit4]   2> 629591 INFO  (coreLoadExecutor-1688-thread-2-processing-x:core1) [    x:collection2] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 629591 INFO  (coreLoadExecutor-1688-thread-2-processing-x:core1) [    x:collection2] o.a.s.c.CoreContainer Creating SolrCore 'collection2' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-002/./collection2, trusted=true
   [junit4]   2> 629591 INFO  (coreLoadExecutor-1688-thread-2-processing-x:core1) [    x:collection2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection2' (registry 'solr.core.collection2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 629592 INFO  (coreLoadExecutor-1688-thread-2-processing-x:core1) [    x:collection2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 629592 INFO  (coreLoadExecutor-1688-thread-2-processing-x:core1) [    x:collection2] o.a.s.c.SolrCore [[collection2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-002/collection2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-002/./collection2/data/]
   [junit4]   2> 629593 INFO  (coreLoadExecutor-1688-thread-1-processing-x:core1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 629593 INFO  (coreLoadExecutor-1688-thread-1-processing-x:core1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-002/./collection1, trusted=true
   [junit4]   2> 629593 INFO  (coreLoadExecutor-1688-thread-3-processing-x:core1) [    x:collection5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 629593 INFO  (coreLoadExecutor-1688-thread-1-processing-x:core1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 629593 INFO  (coreLoadExecutor-1688-thread-1-processing-x:core1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 629593 INFO  (coreLoadExecutor-1688-thread-1-processing-x:core1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-002/./collection1/data/]
   [junit4]   2> 629596 INFO  (coreLoadExecutor-1688-thread-1-processing-x:core1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=20, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 629597 INFO  (coreLoadExecutor-1688-thread-2-processing-x:core1) [    x:collection2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=20, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 629602 INFO  (coreLoadExecutor-1688-thread-3-processing-x:core1) [    x:collection5] o.a.s.s.IndexSchema [collection5] Schema name=tiny
   [junit4]   2> 629628 INFO  (coreLoadExecutor-1688-thread-3-processing-x:core1) [    x:collection5] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
   [junit4]   2> 629628 INFO  (coreLoadExecutor-1688-thread-3-processing-x:core1) [    x:collection5] o.a.s.c.CoreContainer Creating SolrCore 'collection5' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-002/./collection5, trusted=true
   [junit4]   2> 629629 INFO  (coreLoadExecutor-1688-thread-3-processing-x:core1) [    x:collection5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection5' (registry 'solr.core.collection5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@30a67abd
   [junit4]   2> 629629 INFO  (coreLoadExecutor-1688-thread-3-processing-x:core1) [    x:collection5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 629629 INFO  (coreLoadExecutor-1688-thread-3-processing-x:core1) [    x:collection5] o.a.s.c.SolrCore [[collection5] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-002/collection5], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-002/./collection5/data/]
   [junit4]   2> 629631 INFO  (coreLoadExecutor-1688-thread-3-processing-x:core1) [    x:collection5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=20, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 629700 INFO  (coreLoadExecutor-1688-thread-1-processing-x:core1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 629700 INFO  (coreLoadExecutor-1688-thread-1-processing-x:core1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 629702 INFO  (coreLoadExecutor-1688-thread-1-processing-x:core1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=318168228, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 629703 INFO  (coreLoadExecutor-1688-thread-1-processing-x:core1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@9f15366[collection1] main]
   [junit4]   2> 629703 INFO  (coreLoadExecutor-1688-thread-1-processing-x:core1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_500F99E268BE63A1-001/tempDir-002/collection1/conf
   [junit4]   2> 629704 INFO  (coreLoadExecutor-1688-thread-1-processing-x:core1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 629704 INFO  (searcherExecutor-1690-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@9f15366[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 629725 INFO  (coreLoadExecutor-1688-thread-2-processing-x:core1) [    x:collection2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 629725 INFO  (coreLoadExecutor-1688-thread-2-processing-x:core1) [    x:collection2] o.a.s.u.CommitTracker Soft AutoCommit: disab

[...truncated too long message...]

solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection2.shard1.replica_n1, tag=1427030979
   [junit4]   2> 1586060 INFO  (coreCloseExecutor-4672-thread-2) [n:127.0.0.1:35417_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2d8f13e3: rootName = solr_35417, domain = solr.core.testCollection2.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.testCollection2.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@cd5d6a
   [junit4]   2> 1586070 INFO  (coreCloseExecutor-4672-thread-3) [n:127.0.0.1:35417_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.c.SolrCore [testCollection2_shard2_replica_n3]  CLOSING SolrCore org.apache.solr.core.SolrCore@1ddc2f5b
   [junit4]   2> 1586075 INFO  (coreCloseExecutor-4672-thread-3) [n:127.0.0.1:35417_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection2.shard2.replica_n3, tag=500969307
   [junit4]   2> 1586075 INFO  (coreCloseExecutor-4672-thread-3) [n:127.0.0.1:35417_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7279fe1f: rootName = solr_35417, domain = solr.core.testCollection2.shard2.replica_n3, service url = null, agent id = null] for registry solr.core.testCollection2.shard2.replica_n3 / com.codahale.metrics.MetricRegistry@787762a3
   [junit4]   2> 1586082 INFO  (coreCloseExecutor-4672-thread-2) [n:127.0.0.1:35417_solr c:testCollection2 s:shard1 r:core_node2 x:testCollection2_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection2.shard1.leader, tag=1427030979
   [junit4]   2> 1586083 INFO  (coreCloseExecutor-4672-thread-3) [n:127.0.0.1:35417_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection2.shard2.leader, tag=500969307
   [junit4]   2> 1586084 INFO  (jetty-closer-1864-thread-1) [    ] o.a.s.c.Overseer Overseer (id=98607782562168835-127.0.0.1:35417_solr-n_0000000000) closing
   [junit4]   2> 1586084 INFO  (OverseerStateUpdate-98607782562168835-127.0.0.1:35417_solr-n_0000000000) [n:127.0.0.1:35417_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:35417_solr
   [junit4]   2> 1586085 INFO  (jetty-closer-1864-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@44817052{/solr,null,UNAVAILABLE}
   [junit4]   2> 1586086 ERROR (SUITE-LeaderElectionContextKeyTest-seed#[500F99E268BE63A1]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1586086 INFO  (SUITE-LeaderElectionContextKeyTest-seed#[500F99E268BE63A1]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41558 41558
   [junit4]   2> 1586175 INFO  (Thread-2540) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41558 41558
   [junit4]   2> 1586175 WARN  (Thread-2540) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 2 /solr/aliases.json
   [junit4]   2> 2 /solr/clusterprops.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 5 /solr/collections/testCollection2/state.json
   [junit4]   2> 2 /solr/clusterstate.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 2 /solr/live_nodes
   [junit4]   2> 2 /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderElectionContextKeyTest_500F99E268BE63A1-001
   [junit4]   2> Sep 05, 2017 6:16:00 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1447, maxMBSortInHeap=7.784030077557206, sim=RandomSimilarity(queryNorm=false): {}, locale=ro, timezone=Asia/Damascus
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=134199568,total=526909440
   [junit4]   2> NOTE: All tests run in this JVM: [TestUnifiedSolrHighlighter, TestComplexPhraseLeadingWildcard, TestConfigSetsAPIZkFailure, TestCustomStream, TestOrdValues, DocValuesTest, HdfsUnloadDistributedZkTest, LoggingHandlerTest, TestReloadAndDeleteDocs, HighlighterTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, ConvertedLegacyTest, TestSolrConfigHandlerConcurrent, SolrCloudExampleTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestSolrFieldCacheBean, TestNumericTokenStream, TestNumericTerms32, ZkFailoverTest, NoCacheHeaderTest, TestPerFieldSimilarity, TestCloudManagedSchema, TestSolrCloudWithDelegationTokens, ImplicitSnitchTest, TestDistribDocBasedVersion, SOLR749Test, AtomicUpdatesTest, HdfsBasicDistributedZk2Test, TestFuzzyAnalyzedSuggestions, DistribCursorPagingTest, AddBlockUpdateTest, TestPullReplica, TestFieldCacheVsDocValues, EnumFieldTest, QueryParsingTest, JSONWriterTest, BlobRepositoryCloudTest, PropertiesRequestHandlerTest, HdfsChaosMonkeySafeLeaderTest, ShardRoutingTest, StatsReloadRaceTest, TestCloudDeleteByQuery, BitVectorTest, TestUpdate, TestMaxScoreQueryParser, TestScoreJoinQPNoScore, TestHalfAndHalfDocValues, TestCharFilters, TestRequestForwarding, PrimUtilsTest, CacheHeaderTest, MoveReplicaTest, ZkCLITest, TestFieldCacheReopen, TestLegacyFieldReuse, TestSimpleTrackingShardHandler, TestReplicationHandlerBackup, TestTrackingShardHandlerFactory, TestReloadDeadlock, SchemaVersionSpecificBehaviorTest, TestSearchPerf, ForceLeaderTest, BasicAuthIntegrationTest, DOMUtilTest, TestCollectionsAPIViaSolrCloudCluster, TestSolrCLIRunExample, ResourceLoaderTest, DistributedFacetPivotWhiteBoxTest, TestSolrCloudWithHadoopAuthPlugin, ConfigureRecoveryStrategyTest, TestLeaderInitiatedRecoveryThread, TestEmbeddedSolrServerConstructors, BadCopyFieldTest, ConnectionReuseTest, OverseerModifyCollectionTest, CollectionsAPIDistributedZkTest, DistributedFacetPivotLongTailTest, TestClusterStateMutator, ScriptEngineTest, DeleteReplicaTest, TestCoreDiscovery, DataDrivenBlockJoinTest, TestTlogReplica, SortSpecParsingTest, CollectionStateFormat2Test, AlternateDirectoryTest, DistributedQueueTest, TestFiltering, DistributedTermsComponentTest, JsonLoaderTest, SharedFSAutoReplicaFailoverUtilsTest, TestElisionMultitermQuery, CoreMergeIndexesAdminHandlerTest, V2StandaloneTest, TestAnalyzeInfixSuggestions, TestRawResponseWriter, TestConfigsApi, TestHdfsCloudBackupRestore, ExitableDirectoryReaderTest, ClassificationUpdateProcessorTest, HLLSerializationTest, TestDocBasedVersionConstraints, HdfsLockFactoryTest, HdfsSyncSliceTest, SubstringBytesRefFilterTest, TestFastLRUCache, VersionInfoTest, ZkControllerTest, TestSolrIndexConfig, SolrCmdDistributorTest, LeaderInitiatedRecoveryOnCommitTest, TestDistributedGrouping, LeaderElectionContextKeyTest]
   [junit4] Completed [725/732 (2!)] on J1 in 7.11s, 1 test, 1 failure <<< FAILURES!

[...truncated 15492 lines...]
-check-forbidden-all:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.8
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.8
[forbidden-apis] Reading bundled API signatures: jdk-non-portable
[forbidden-apis] Reading bundled API signatures: jdk-reflection
[forbidden-apis] Reading API signatures: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/tools/forbiddenApis/lucene.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning classes for violations...
[forbidden-apis] Forbidden method invocation: java.lang.Character#codePointBefore(char[],int) [Implicit start offset is error-prone when the char[] is a buffer and the first chars are random chars]
[forbidden-apis]   in org.apache.lucene.analysis.icu.ICUNormalizer2CharFilter (ICUNormalizer2CharFilter.java:113)
[forbidden-apis] Scanned 69 (and 374 related) class file(s) for forbidden API invocations (in 0.05s), 1 error(s).

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:810: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:117: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build.xml:119: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:2205: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/analysis/build.xml:157: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/analysis/build.xml:38: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:2407: Check for forbidden API calls failed, see log.

Total time: 96 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2090 - Still unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2090/

1 tests failed.
FAILED:  org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest.testConcurrentUpdate

Error Message:
Expected CUSS to send 500 but got 107

Stack Trace:
java.lang.AssertionError: Expected CUSS to send 500 but got 107
        at __randomizedtesting.SeedInfo.seed([BC878B6437711742:4481D29FE3497209]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest.testConcurrentUpdate(ConcurrentUpdateSolrClientTest.java:189)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 14162 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest
   [junit4]   2> 0    INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/init-core-data-001
   [junit4]   2> 804  INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1216 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.e.j.u.log Logging initialized @17055ms
   [junit4]   2> 1267 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 4052 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 4052 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 4065 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-002/cores/core
   [junit4]   2> 5530 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 5724 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@57f8a370{/solr,null,AVAILABLE}
   [junit4]   2> 5876 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@39c1a34e{SSL,[ssl, http/1.1]}{127.0.0.1:33983}
   [junit4]   2> 5876 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.e.j.s.Server Started @21748ms
   [junit4]   2> 5876 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33983, configSetBaseDir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-002/cores}
   [junit4]   2> 6284 ERROR (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 6284 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 6300 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 6300 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 6433 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-06T16:45:33.883Z
   [junit4]   2> 6656 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.c.SolrResourceLoader Using system property solr.solr.home: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001
   [junit4]   2> 6704 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/solr.xml
   [junit4]   2> 12206 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001
   [junit4]   2> 12207 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 13548 WARN  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../contrib/extraction/lib
   [junit4]   2> 13561 WARN  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../dist
   [junit4]   2> 13562 WARN  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../contrib/clustering/lib
   [junit4]   2> 13562 WARN  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../dist
   [junit4]   2> 13562 WARN  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../contrib/langid/lib
   [junit4]   2> 13563 WARN  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../dist
   [junit4]   2> 13563 WARN  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../contrib/ltr/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../contrib/ltr/lib
   [junit4]   2> 13563 WARN  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../dist filtered by solr-ltr-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../dist
   [junit4]   2> 13563 WARN  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../contrib/velocity/lib
   [junit4]   2> 13564 WARN  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/../../../../dist
   [junit4]   2> 13897 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.SolrIndexConfig IndexWriter infoStream solr logging is enabled
   [junit4]   2> 14002 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 15758 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=example
   [junit4]   2> 29940 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.rest.schema.analysis.ManagedWordSetResource for path /schema/analysis/stopwords/english
   [junit4]   2> 29942 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.rest.schema.analysis.ManagedSynonymGraphFilterFactory$SynonymManager for path /schema/analysis/synonyms/english
   [junit4]   2> 29942 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedStopFilterFactory to existing ManagedResource /schema/analysis/stopwords/english
   [junit4]   2> 29942 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedSynonymGraphFilterFactory to existing ManagedResource /schema/analysis/synonyms/english
   [junit4]   2> 30057 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema example/1.6 with uniqueid field id
   [junit4]   2> 30059 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedStopFilterFactory to existing ManagedResource /schema/analysis/stopwords/english
   [junit4]   2> 30059 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedSynonymGraphFilterFactory to existing ManagedResource /schema/analysis/synonyms/english
   [junit4]   2> 30059 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedStopFilterFactory to existing ManagedResource /schema/analysis/stopwords/english
   [junit4]   2> 30059 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedSynonymGraphFilterFactory to existing ManagedResource /schema/analysis/synonyms/english
   [junit4]   2> 30581 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1, trusted=true
   [junit4]   2> 31274 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 31355 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/data/]
   [junit4]   2> 32246 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-6-thread-1]: init: current segments file is "segments"; deletionPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper@3064daa6
   [junit4]   2> 32260 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-6-thread-1]: now checkpoint "" [0 segments ; isCommit = false]
   [junit4]   2> 32260 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-6-thread-1]: 0 msec to checkpoint
   [junit4]   2> 32276 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: init: create=true
   [junit4]   2> 32276 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]:
   [junit4]   2> dir=MockDirectoryWrapper(RAMDirectory@2b652bc4 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@36aff6e4)
   [junit4]   2> index=
   [junit4]   2> version=8.0.0
   [junit4]   2> analyzer=org.apache.solr.update.SolrIndexConfig$DelayedSchemaAnalyzer
   [junit4]   2> ramBufferSizeMB=100.0
   [junit4]   2> maxBufferedDocs=-1
   [junit4]   2> mergedSegmentWarmer=null
   [junit4]   2> delPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper
   [junit4]   2> commit=null
   [junit4]   2> openMode=CREATE
   [junit4]   2> similarity=org.apache.solr.search.similarities.SchemaSimilarityFactory$SchemaSimilarity
   [junit4]   2> mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=-1, maxMergeCount=-1, ioThrottle=true
   [junit4]   2> codec=Lucene70
   [junit4]   2> infoStream=org.apache.solr.update.LoggingInfoStream
   [junit4]   2> mergePolicy=[TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=5120.0, floorSegmentMB=2.0, forceMergeDeletesPctAllowed=10.0, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1
   [junit4]   2> indexerThreadPool=org.apache.lucene.index.DocumentsWriterPerThreadPool@614634e5
   [junit4]   2> readerPooling=true
   [junit4]   2> perThreadHardLimitMB=1945
   [junit4]   2> useCompoundFile=false
   [junit4]   2> commitOnClose=true
   [junit4]   2> indexSort=null
   [junit4]   2> writer=org.apache.solr.update.SolrIndexWriter@63ff0802
   [junit4]   2>
   [junit4]   2> 32406 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: MMapDirectory.UNMAP_SUPPORTED=true
   [junit4]   2> 32407 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: now flush at close
   [junit4]   2> 32407 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]:   start flush: applyAllDeletes=true
   [junit4]   2> 32407 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]:   index before flush
   [junit4]   2> 32407 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreLoadExecutor-6-thread-1]: startFullFlush
   [junit4]   2> 32407 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreLoadExecutor-6-thread-1]: coreLoadExecutor-6-thread-1 finishFullFlush success=true
   [junit4]   2> 32407 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: now apply all deletes for all segments buffered updates bytesUsed=0 reader pool bytesUsed=0
   [junit4]   2> 32408 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][coreLoadExecutor-6-thread-1]: waitApply: no deletes to apply
   [junit4]   2> 32424 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreLoadExecutor-6-thread-1]: initDynamicDefaults spins=false maxThreadCount=1 maxMergeCount=6
   [junit4]   2> 32424 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreLoadExecutor-6-thread-1]: updateMergeThreads ioThrottle=true targetMBPerSec=10240.0 MB/sec
   [junit4]   2> 32425 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreLoadExecutor-6-thread-1]: now merge
   [junit4]   2> 32425 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreLoadExecutor-6-thread-1]:   index:
   [junit4]   2> 32425 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreLoadExecutor-6-thread-1]:   no more merges pending; now return
   [junit4]   2> 32425 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: waitForMerges
   [junit4]   2> 32425 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: waitForMerges done
   [junit4]   2> 32425 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: commit: start
   [junit4]   2> 32425 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: commit: enter lock
   [junit4]   2> 32425 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: commit: now prepare
   [junit4]   2> 32425 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: prepareCommit: flush
   [junit4]   2> 32425 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]:   index before flush
   [junit4]   2> 32425 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreLoadExecutor-6-thread-1]: startFullFlush
   [junit4]   2> 32425 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: now apply all deletes for all segments buffered updates bytesUsed=0 reader pool bytesUsed=0
   [junit4]   2> 32426 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][coreLoadExecutor-6-thread-1]: waitApply: no deletes to apply
   [junit4]   2> 32426 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreLoadExecutor-6-thread-1]: coreLoadExecutor-6-thread-1 finishFullFlush success=true
   [junit4]   2> 32426 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: startCommit(): start
   [junit4]   2> 32426 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: startCommit index= changeCount=2
   [junit4]   2> 32538 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: startCommit: wrote pending segments file "pending_segments_1"
   [junit4]   2> 32539 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: done all syncs: []
   [junit4]   2> 32539 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: commit: pendingCommit != null
   [junit4]   2> 32540 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: commit: done writing segments file "segments_1"
   [junit4]   2> 32584 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-6-thread-1]: now checkpoint "" [0 segments ; isCommit = true]
   [junit4]   2> 32636 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-6-thread-1]: 95 msec to checkpoint
   [junit4]   2> 32636 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: commit: took 210.5 msec
   [junit4]   2> 32636 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: commit: done
   [junit4]   2> 32636 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: rollback
   [junit4]   2> 32637 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: all running merges have aborted
   [junit4]   2> 32637 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: rollback: done finish merges
   [junit4]   2> 32637 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreLoadExecutor-6-thread-1]: abort
   [junit4]   2> 32637 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreLoadExecutor-6-thread-1]: done abort success=true
   [junit4]   2> 32666 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: rollback: infos=
   [junit4]   2> 32666 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-6-thread-1]: now checkpoint "" [0 segments ; isCommit = false]
   [junit4]   2> 32666 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-6-thread-1]: 0 msec to checkpoint
   [junit4]   2> 32850 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5
   [junit4]   2> 39066 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 39066 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 39146 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
   [junit4]   2> 39146 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 39203 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-6-thread-1]: init: current segments file is "segments_1"; deletionPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper@3064daa6
   [junit4]   2> 39203 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-6-thread-1]: init: load commit "segments_1"
   [junit4]   2> 39223 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-6-thread-1]: now checkpoint "" [0 segments ; isCommit = false]
   [junit4]   2> 39223 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-6-thread-1]: 0 msec to checkpoint
   [junit4]   2> 39223 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: init: create=false
   [junit4]   2> 39224 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]:
   [junit4]   2> dir=MockDirectoryWrapper(RAMDirectory@2b652bc4 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@36aff6e4)
   [junit4]   2> index=
   [junit4]   2> version=8.0.0
   [junit4]   2> analyzer=org.apache.solr.update.SolrIndexConfig$DelayedSchemaAnalyzer
   [junit4]   2> ramBufferSizeMB=100.0
   [junit4]   2> maxBufferedDocs=-1
   [junit4]   2> mergedSegmentWarmer=null
   [junit4]   2> delPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper
   [junit4]   2> commit=null
   [junit4]   2> openMode=APPEND
   [junit4]   2> similarity=org.apache.solr.search.similarities.SchemaSimilarityFactory$SchemaSimilarity
   [junit4]   2> mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=-1, maxMergeCount=-1, ioThrottle=true
   [junit4]   2> codec=Lucene70
   [junit4]   2> infoStream=org.apache.solr.update.LoggingInfoStream
   [junit4]   2> mergePolicy=[TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=5120.0, floorSegmentMB=2.0, forceMergeDeletesPctAllowed=10.0, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1
   [junit4]   2> indexerThreadPool=org.apache.lucene.index.DocumentsWriterPerThreadPool@372a421f
   [junit4]   2> readerPooling=true
   [junit4]   2> perThreadHardLimitMB=1945
   [junit4]   2> useCompoundFile=false
   [junit4]   2> commitOnClose=true
   [junit4]   2> indexSort=null
   [junit4]   2> writer=org.apache.solr.update.SolrIndexWriter@422e455d
   [junit4]   2>
   [junit4]   2> 39224 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: MMapDirectory.UNMAP_SUPPORTED=true
   [junit4]   2> 39226 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: flush at getReader
   [junit4]   2> 39227 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreLoadExecutor-6-thread-1]: startFullFlush
   [junit4]   2> 39227 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: now apply all deletes for all segments buffered updates bytesUsed=0 reader pool bytesUsed=0
   [junit4]   2> 39227 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][coreLoadExecutor-6-thread-1]: waitApply: no deletes to apply
   [junit4]   2> 39307 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: incRefDeleter for NRT reader version=2 segments=
   [junit4]   2> 39321 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: return reader version=2 reader=StandardDirectoryReader(segments_1:2:nrt)
   [junit4]   2> 39321 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreLoadExecutor-6-thread-1]: coreLoadExecutor-6-thread-1 finishFullFlush success=true
   [junit4]   2> 39321 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-6-thread-1]: getReader took 95 msec
   [junit4]   2> 39555 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@76337f4f[collection1] main]
   [junit4]   2> 39722 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/conf
   [junit4]   2> 39764 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Loaded LinkedHashMap at path _rest_managed.json using file:dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/conf
   [junit4]   2> 39764 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.ManagedResource Loaded initArgs {} for /rest/managed
   [junit4]   2> 39765 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Loaded LinkedHashMap at path _schema_analysis_stopwords_english.json using file:dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/conf
   [junit4]   2> 39765 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.ManagedResource Loaded initArgs {ignoreCase=true} for /schema/analysis/stopwords/english
   [junit4]   2> 39765 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.s.a.ManagedWordSetResource Loaded 35 words for /schema/analysis/stopwords/english
   [junit4]   2> 39765 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.ManagedResource Notified 4 observers of /schema/analysis/stopwords/english
   [junit4]   2> 39766 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Loaded LinkedHashMap at path _schema_analysis_synonyms_english.json using file:dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/conf
   [junit4]   2> 39766 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.ManagedResource Loaded initArgs {ignoreCase=true,format=solr} for /schema/analysis/synonyms/english
   [junit4]   2> 39783 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.s.a.ManagedSynonymGraphFilterFactory Loaded 3 synonym mappings for /schema/analysis/synonyms/english
   [junit4]   2> 39786 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.r.ManagedResource Notified 4 observers of /schema/analysis/synonyms/english
   [junit4]   2> 40110 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 40262 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=default,field=text,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01}
   [junit4]   2> 40582 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.h.c.SuggestComponent Initializing SuggestComponent
   [junit4]   2> 40616 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.s.s.SolrSuggester init: {name=mySuggester,lookupImpl=FuzzyLookupFactory,dictionaryImpl=DocumentDictionaryFactory,field=cat,weightField=price,suggestAnalyzerFieldType=string,buildOnStartup=false}
   [junit4]   2> 41069 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.s.s.SolrSuggester Dictionary loaded with params: {name=mySuggester,lookupImpl=FuzzyLookupFactory,dictionaryImpl=DocumentDictionaryFactory,field=cat,weightField=price,suggestAnalyzerFieldType=string,buildOnStartup=false}
   [junit4]   2> 41117 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.h.c.QueryElevationComponent Loading QueryElevation from: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001/tempDir-001/collection1/conf/elevate.xml
   [junit4]   2> 41232 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 42002 INFO  (searcherExecutor-7-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@76337f4f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 42134 INFO  (coreLoadExecutor-6-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577809472091324416
   [junit4]   2> 42608 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.SolrJettyTestBase Jetty Assigned Port#33983
   [junit4]   2> 42638 INFO  (TEST-ConcurrentUpdateSolrClientTest.testConcurrentUpdate-seed#[BC878B6437711742]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testConcurrentUpdate
   [junit4]   2> 52486 INFO  (searcherExecutor-7-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false&event=firstSearcher} hits=0 status=0 QTime=10434
   [junit4]   2> 52501 INFO  (searcherExecutor-7-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.QuerySenderListener QuerySenderListener done.
   [junit4]   2> 52502 INFO  (searcherExecutor-7-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 52519 INFO  (searcherExecutor-7-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 52705 INFO  (searcherExecutor-7-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@76337f4f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 57324 INFO  (qtp1075508131-25) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@2b4fb2c4 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Shutting down, pool size = 2, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@4b604d8b rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 57634 INFO  (TEST-ConcurrentUpdateSolrClientTest.testConcurrentUpdate-seed#[BC878B6437711742]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testConcurrentUpdate
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@fdfcb24 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@1e075a64 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@21f3baa5 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@2ebfa2bc rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@1562574f rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@e262ac1 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@5544d929 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Shutting down, pool size = 2, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@4615221c rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@327bfbf7 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@53dcb516 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@9a29817 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@bf8e75d rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Shutting down, pool size = 2, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@1363ec3a rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Shutting down, pool size = 2, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@7fc2328b rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@35948b31 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@68ddb4af rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@f8ce6eb rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@7f53a4f5 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@1a1e3e5b rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@2f2dc6dc rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@56459859 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@46428b36 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@e95bc55 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@46e8f716 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@542fc6f4 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@2e82bd8a rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@4d573a07 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:527)
   [junit4]   2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClientTest$SendDocsRunnable.run(ConcurrentUpdateSolrClientTest.java:300)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$121/2235623@766543f3 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@65644026[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 2]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:173)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.addRunner(ConcurrentUpdateSolrClient.java:429)
   [junit4]   2> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrCl

[...truncated too long message...]

8.0.0):C420) Uninverting(_2(8.0.0):C80)))}
   [junit4]   2> 67642 INFO  (searcherExecutor-7-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][searcherExecutor-7-thread-1-processing-x:collection1]: delete [_0.si, _0_Lucene50_0.doc, _0_Lucene50_0.tim, _0_Lucene70_0.dvd, _0.fdx, _0_Lucene50_0.tip, _0.fdt, _0_Lucene70_0.dvm, _0.fnm]
   [junit4]   2> 67642 INFO  (searcherExecutor-7-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][searcherExecutor-7-thread-1-processing-x:collection1]: decRefDeleter for NRT reader version=6 segments=_0(8.0.0):C1
   [junit4]   2> 67642 INFO  (qtp1075508131-21) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 518
   [junit4]   2> 67675 INFO  (qtp1075508131-24) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=500 status=0 QTime=29
   [junit4]   2> 68132 INFO  (qtp1075508131-22) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=500 status=0 QTime=0
   [junit4]   2> 68133 INFO  (TEST-ConcurrentUpdateSolrClientTest.testConcurrentCollectionUpdate-seed#[BC878B6437711742]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testConcurrentCollectionUpdate
   [junit4]   2> 68207 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@39c1a34e{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 68207 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=598033268
   [junit4]   2> 68211 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 68211 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 68211 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 68248 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@686741f3
   [junit4]   2> 68248 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1751597555
   [junit4]   2> 68249 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: now flush at close
   [junit4]   2> 68249 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]:   start flush: applyAllDeletes=true
   [junit4]   2> 68249 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]:   index before flush _1(8.0.0):C420 _2(8.0.0):C80
   [junit4]   2> 68249 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-12-thread-1]: startFullFlush
   [junit4]   2> 68249 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-12-thread-1]: coreCloseExecutor-12-thread-1 finishFullFlush success=true
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: now apply all deletes for all segments buffered updates bytesUsed=0 reader pool bytesUsed=0
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-12-thread-1]: waitApply: no deletes to apply
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-12-thread-1]: updateMergeThreads ioThrottle=true targetMBPerSec=10240.0 MB/sec
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-12-thread-1]: now merge
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-12-thread-1]:   index: _1(8.0.0):C420 _2(8.0.0):C80
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-12-thread-1]:   no more merges pending; now return
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: waitForMerges
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: waitForMerges done
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: commit: start
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: commit: enter lock
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: commit: now prepare
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: prepareCommit: flush
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]:   index before flush _1(8.0.0):C420 _2(8.0.0):C80
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-12-thread-1]: startFullFlush
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: now apply all deletes for all segments buffered updates bytesUsed=0 reader pool bytesUsed=0
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-12-thread-1]: waitApply: no deletes to apply
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-12-thread-1]: coreCloseExecutor-12-thread-1 finishFullFlush success=true
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: startCommit(): start
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]:   skip startCommit(): no changes pending
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: commit: pendingCommit == null; skip
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: commit: took 0.4 msec
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: commit: done
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: rollback
   [junit4]   2> 68250 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: all running merges have aborted
   [junit4]   2> 68251 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: rollback: done finish merges
   [junit4]   2> 68251 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-12-thread-1]: abort
   [junit4]   2> 68251 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-12-thread-1]: done abort success=true
   [junit4]   2> 68251 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-12-thread-1]: rollback: infos=_1(8.0.0):C420 _2(8.0.0):C80
   [junit4]   2> 68251 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-12-thread-1]: now checkpoint "_1(8.0.0):C420 _2(8.0.0):C80" [2 segments ; isCommit = false]
   [junit4]   2> 68251 INFO  (coreCloseExecutor-12-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-12-thread-1]: 0 msec to checkpoint
   [junit4]   2> 68362 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[BC878B6437711742]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@57f8a370{/solr,null,UNAVAILABLE}
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_BC878B6437711742-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1904, maxMBSortInHeap=6.716714123972825, sim=RandomSimilarity(queryNorm=false): {}, locale=sr-RS, timezone=America/Chicago
   [junit4]   2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=168631264,total=323485696
   [junit4]   2> NOTE: All tests run in this JVM: [StreamExpressionToExpessionTest, LengthEvaluatorTest, ConcurrentUpdateSolrClientTest]
   [junit4] Completed [11/145 (1!)] on J0 in 68.70s, 3 tests, 1 failure <<< FAILURES!

[...truncated 45645 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2091 - Failure

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2091/

128 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.analytics.facet.FieldFacetExtrasCloudTest

Error Message:
Error starting up MiniSolrCloudCluster

Stack Trace:
java.lang.Exception: Error starting up MiniSolrCloudCluster
        at __randomizedtesting.SeedInfo.seed([A67DD776F0F293C2]:0)
        at org.apache.solr.cloud.MiniSolrCloudCluster.checkForExceptions(MiniSolrCloudCluster.java:507)
        at org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:251)
        at org.apache.solr.cloud.SolrCloudTestCase$Builder.configure(SolrCloudTestCase.java:190)
        at org.apache.solr.analytics.facet.AbstractAnalyticsFacetCloudTest.setupCluster(AbstractAnalyticsFacetCloudTest.java:56)
        at org.apache.solr.analytics.facet.FieldFacetExtrasCloudTest.beforeClass(FieldFacetExtrasCloudTest.java:47)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
        Suppressed: java.lang.RuntimeException: java.lang.NullPointerException
                at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1600)
                at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1659)
                at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1316)
                at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1145)
                at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
                at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
                at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
                at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
                at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
                at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
                at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:384)
                at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:247)
                at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                ... 1 more
        Caused by: java.lang.NullPointerException
                at org.apache.solr.util.configuration.providers.EnvSSLCredentialProvider.getCredential(EnvSSLCredentialProvider.java:57)
                at org.apache.solr.util.configuration.providers.AbstractSSLCredentialProvider.getCredential(AbstractSSLCredentialProvider.java:53)
                at org.apache.solr.util.configuration.SSLConfigurations.getPassword(SSLConfigurations.java:123)
                at org.apache.solr.util.configuration.SSLConfigurations.getClientKeyStorePassword(SSLConfigurations.java:109)
                at org.apache.solr.util.configuration.SSLConfigurations.init(SSLConfigurations.java:62)
                at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:148)
                at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139)
                at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
                at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1596)
                ... 16 more
        Suppressed: java.lang.RuntimeException: java.lang.NullPointerException
                at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1600)
                at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1659)
                at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1316)
                at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1145)
                at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
                at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
                at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
                at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
                at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
                at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
                at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:384)
                at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:247)
                at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                ... 1 more
        Caused by: java.lang.NullPointerException
                at org.apache.solr.util.configuration.providers.EnvSSLCredentialProvider.getCredential(EnvSSLCredentialProvider.java:57)
                at org.apache.solr.util.configuration.providers.AbstractSSLCredentialProvider.getCredential(AbstractSSLCredentialProvider.java:53)
                at org.apache.solr.util.configuration.SSLConfigurations.getPassword(SSLConfigurations.java:123)
                at org.apache.solr.util.configuration.SSLConfigurations.getClientKeyStorePassword(SSLConfigurations.java:109)
                at org.apache.solr.util.configuration.SSLConfigurations.init(SSLConfigurations.java:62)
                at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:148)
                at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139)
                at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
                at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1596)
                ... 16 more


FAILED:  junit.framework.TestSuite.org.apache.solr.analytics.facet.FieldFacetExtrasCloudTest

Error Message:
18 threads leaked from SUITE scope at org.apache.solr.analytics.facet.FieldFacetExtrasCloudTest:     1) Thread[id=37, name=qtp811132554-37, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)    2) Thread[id=49, name=qtp1303983303-49, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)    3) Thread[id=38, name=qtp811132554-38, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)    4) Thread[id=41, name=qtp811132554-41, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)    5) Thread[id=51, name=org.eclipse.jetty.server.session.HashSessionManager@522fa32bTimer, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    6) Thread[id=36, name=qtp811132554-36-acceptor-0@2fc53435-ServerConnector@52b3dca5{HTTP/1.1,[http/1.1]}{127.0.0.1:37952}, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)    7) Thread[id=34, name=qtp811132554-34, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)    8) Thread[id=44, name=qtp1303983303-44, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)    9) Thread[id=42, name=org.eclipse.jetty.server.session.HashSessionManager@49cb34a8Timer, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   10) Thread[id=35, name=qtp811132554-35, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   11) Thread[id=47, name=qtp1303983303-47, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   12) Thread[id=50, name=qtp1303983303-50, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   13) Thread[id=40, name=qtp811132554-40, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   14) Thread[id=48, name=qtp1303983303-48, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   15) Thread[id=45, name=qtp1303983303-45-acceptor-0@125d309d-ServerConnector@299d83bc{HTTP/1.1,[http/1.1]}{127.0.0.1:36120}, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   16) Thread[id=43, name=qtp1303983303-43, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   17) Thread[id=46, name=qtp1303983303-46, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   18) Thread[id=39, name=qtp811132554-39, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 18 threads leaked from SUITE scope at org.apache.solr.analytics.facet.FieldFacetExtrasCloudTest:
   1) Thread[id=37, name=qtp811132554-37, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
   2) Thread[id=49, name=qtp1303983303-49, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
   3) Thread[id=38, name=qtp811132554-38, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
   4) Thread[id=41, name=qtp811132554-41, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
   5) Thread[id=51, name=org.eclipse.jetty.server.session.HashSessionManager@522fa32bTimer, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   6) Thread[id=36, name=qtp811132554-36-acceptor-0@2fc53435-ServerConnector@52b3dca5{HTTP/1.1,[http/1.1]}{127.0.0.1:37952}, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
   7) Thread[id=34, name=qtp811132554-34, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
        at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
   8) Thread[id=44, name=qtp1303983303-44, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
        at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
   9) Thread[id=42, name=org.eclipse.jetty.server.session.HashSessionManager@49cb34a8Timer, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  10) Thread[id=35, name=qtp811132554-35, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
        at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
  11) Thread[id=47, name=qtp1303983303-47, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
  12) Thread[id=50, name=qtp1303983303-50, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
  13) Thread[id=40, name=qtp811132554-40, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
  14) Thread[id=48, name=qtp1303983303-48, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
  15) Thread[id=45, name=qtp1303983303-45-acceptor-0@125d309d-ServerConnector@299d83bc{HTTP/1.1,[http/1.1]}{127.0.0.1:36120}, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
  16) Thread[id=43, name=qtp1303983303-43, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
        at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
  17) Thread[id=46, name=qtp1303983303-46, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
  18) Thread[id=39, name=qtp811132554-39, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([A67DD776F0F293C2]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.analytics.facet.FieldFacetExtrasCloudTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=43, name=qtp1303983303-43, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]         at sun.nio.ch.IOUtil.drain(Native Method)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:104)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)    2) Thread[id=51, name=org.eclipse.jetty.server.session.HashSessionManager@522fa32bTimer, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    3) Thread[id=34, name=qtp811132554-34, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]         at sun.nio.ch.EPollArrayWrapper.interrupt(Native Method)         at sun.nio.ch.EPollArrayWrapper.interrupt(EPollArrayWrapper.java:317)         at sun.nio.ch.EPollSelectorImpl.wakeup(EPollSelectorImpl.java:207)         at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)         at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)    4) Thread[id=44, name=qtp1303983303-44, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)    5) Thread[id=42, name=org.eclipse.jetty.server.session.HashSessionManager@49cb34a8Timer, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    6) Thread[id=35, name=qtp811132554-35, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]         at sun.nio.ch.EPollArrayWrapper.interrupt(Native Method)         at sun.nio.ch.EPollArrayWrapper.interrupt(EPollArrayWrapper.java:317)         at sun.nio.ch.EPollSelectorImpl.wakeup(EPollSelectorImpl.java:207)         at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)         at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=43, name=qtp1303983303-43, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.nio.ch.IOUtil.drain(Native Method)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:104)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
        at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
   2) Thread[id=51, name=org.eclipse.jetty.server.session.HashSessionManager@522fa32bTimer, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   3) Thread[id=34, name=qtp811132554-34, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.nio.ch.EPollArrayWrapper.interrupt(Native Method)
        at sun.nio.ch.EPollArrayWrapper.interrupt(EPollArrayWrapper.java:317)
        at sun.nio.ch.EPollSelectorImpl.wakeup(EPollSelectorImpl.java:207)
        at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)
        at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
        at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
   4) Thread[id=44, name=qtp1303983303-44, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
        at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
   5) Thread[id=42, name=org.eclipse.jetty.server.session.HashSessionManager@49cb34a8Timer, state=TIMED_WAITING, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   6) Thread[id=35, name=qtp811132554-35, state=RUNNABLE, group=TGRP-FieldFacetExtrasCloudTest]
        at sun.nio.ch.EPollArrayWrapper.interrupt(Native Method)
        at sun.nio.ch.EPollArrayWrapper.interrupt(EPollArrayWrapper.java:317)
        at sun.nio.ch.EPollSelectorImpl.wakeup(EPollSelectorImpl.java:207)
        at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)
        at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
        at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([A67DD776F0F293C2]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.analytics.facet.RangeFacetCloudTest

Error Message:
Error starting up MiniSolrCloudCluster

Stack Trace:
java.lang.Exception: Error starting up MiniSolrCloudCluster
        at __randomizedtesting.SeedInfo.seed([A67DD776F0F293C2]:0)
        at org.apache.solr.cloud.MiniSolrCloudCluster.checkForExceptions(MiniSolrCloudCluster.java:507)
        at org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:251)
        at org.apache.solr.cloud.SolrCloudTestCase$Builder.configure(SolrCloudTestCase.java:190)
        at org.apache.solr.analytics.facet.AbstractAnalyticsFacetCloudTest.setupCluster(AbstractAnalyticsFacetCloudTest.java:56)
        at org.apache.solr.analytics.facet.RangeFacetCloudTest.beforeClass(RangeFacetCloudTest.java:49)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
        Suppressed: java.lang.RuntimeException: java.lang.NullPointerException
                at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1600)
                at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1659)
                at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1316)
                at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1145)
                at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
                at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
                at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
                at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
                at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
                at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
                at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:384)
                at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:247)
                at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                ... 1 more
        Caused by: java.lang.NullPointerException
                at org.apache.solr.util.configuration.providers.EnvSSLCredentialProvider.getCredential(EnvSSLCredentialProvider.java:57)
                at org.apache.solr.util.configuration.providers.AbstractSSLCredentialProvider.getCredential(AbstractSSLCredentialProvider.java:53)
                at org.apache.solr.util.configuration.SSLConfigurations.getPassword(SSLConfigurations.java:123)
                at org.apache.solr.util.configuration.SSLConfigurations.getClientKeyStorePassword(SSLConfigurations.java:109)
                at org.apache.solr.util.configuration.SSLConfigurations.init(SSLConfigurations.java:62)
                at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:148)
                at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139)
                at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
                at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1596)
                ... 16 more
        Suppressed: java.lang.RuntimeException: java.lang.NullPointerException
                at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1600)
                at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1659)
                at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1316)
                at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1145)
                at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
                at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
                at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
                at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
                at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
                at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
                at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:384)
                at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:247)
                at java.util.concurrent.FutureTask.run(FutureTask.java:266)
                at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                ... 1 more
        Caused by: java.lang.NullPointerException
                at org.apache.solr.util.configuration.providers.EnvSSLCredentialProvider.getCredential(EnvSSLCredentialProvider.java:57)
                at org.apache.solr.util.configuration.providers.AbstractSSLCredentialProvider.getCredential(AbstractSSLCredentialProvider.java:53)
                at org.apache.solr.util.configuration.SSLConfigurations.getPassword(SSLConfigurations.java:123)
                at org.apache.solr.util.configuration.SSLConfigurations.getClientKeyStorePassword(SSLConfigurations.java:109)
                at org.apache.solr.util.configuration.SSLConfigurations.init(SSLConfigurations.java:62)
                at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:148)
                at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139)
                at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
                at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1596)
                ... 16 more


FAILED:  junit.framework.TestSuite.org.apache.solr.analytics.facet.RangeFacetCloudTest

Error Message:
18 threads leaked from SUITE scope at org.apache.solr.analytics.facet.RangeFacetCloudTest:     1) Thread[id=32, name=org.eclipse.jetty.server.session.HashSessionManager@26ba244fTimer, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    2) Thread[id=25, name=qtp565295732-25, state=RUNNABLE, group=TGRP-RangeFacetCloudTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)    3) Thread[id=27, name=qtp565295732-27, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)    4) Thread[id=28, name=qtp565295732-28, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)    5) Thread[id=41, name=org.eclipse.jetty.server.session.HashSessionManager@197f3e95Timer, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    6) Thread[id=29, name=qtp565295732-29, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)    7) Thread[id=36, name=qtp222076581-36, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)    8) Thread[id=31, name=qtp565295732-31, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)    9) Thread[id=30, name=qtp565295732-30, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   10) Thread[id=35, name=qtp222076581-35-acceptor-0@63292179-ServerConnector@69e54e58{HTTP/1.1,[http/1.1]}{127.0.0.1:46553}, state=RUNNABLE, group=TGRP-RangeFacetCloudTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   11) Thread[id=34, name=qtp222076581-34, state=RUNNABLE, group=TGRP-RangeFacetCloudTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   12) Thread[id=38, name=qtp222076581-38, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   13) Thread[id=39, name=qtp222076581-39, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   14) Thread[id=24, name=qtp565295732-24, state=RUNNABLE, group=TGRP-RangeFacetCloudTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   15) Thread[id=26, name=qtp565295732-26-acceptor-0@46aab256-ServerConnector@6550bb82{HTTP/1.1,[http/1.1]}{127.0.0.1:46568}, state=RUNNABLE, group=TGRP-RangeFacetCloudTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   16) Thread[id=40, name=qtp222076581-40, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   17) Thread[id=33, name=qtp222076581-33, state=RUNNABLE, group=TGRP-RangeFacetCloudTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   18) Thread[id=37, name=qtp222076581-37, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 18 threads leaked from SUITE scope at org.apache.solr.analytics.facet.RangeFacetCloudTest:
   1) Thread[id=32, name=org.eclipse.jetty.server.session.HashSessionManager@26ba244fTimer, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   2) Thread[id=25, name=qtp565295732-25, state=RUNNABLE, group=TGRP-RangeFacetCloudTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
        at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
   3) Thread[id=27, name=qtp565295732-27, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
   4) Thread[id=28, name=qtp565295732-28, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
   5) Thread[id=41, name=org.eclipse.jetty.server.session.HashSessionManager@197f3e95Timer, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   6) Thread[id=29, name=qtp565295732-29, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
   7) Thread[id=36, name=qtp222076581-36, state=TIMED_WAITING, group=TGRP-RangeFacetCloudTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.

[...truncated too long message...]

llArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
   [junit4]    >         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
   [junit4]    >         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]    >         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    5) Thread[id=147, name=org.eclipse.jetty.server.session.HashSessionManager@26dd4c1bTimer, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    6) Thread[id=130, name=qtp1294663496-130, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
   [junit4]    >         at sun.nio.ch.IOUtil.drain(Native Method)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:104)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
   [junit4]    >         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
   [junit4]    >         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]    >         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    7) Thread[id=129, name=org.eclipse.jetty.server.session.HashSessionManager@423be431Timer, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    8) Thread[id=131, name=qtp1294663496-131, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.interrupt(Native Method)
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.interrupt(EPollArrayWrapper.java:317)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.wakeup(EPollSelectorImpl.java:207)
   [junit4]    >         at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)
   [junit4]    >         at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)
   [junit4]    >         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
   [junit4]    >         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
   [junit4]    >         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]    >         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    9) Thread[id=138, name=org.eclipse.jetty.server.session.HashSessionManager@48dfa242Timer, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([750DE4C2290A8051]:0)
   [junit4] Completed [33/33 (26!)] on J2 in 27.62s, 1 test, 3 errors <<< FAILURES!

[...truncated 38721 lines...]
-documentation-lint:
    [jtidy] Checking for broken html (such as invalid tags)...
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/jtidy_tmp
     [echo] Checking for broken links...
     [exec]
     [exec] Crawl/parse...
     [exec]
     [exec] Verify...
     [echo] Checking for malformed docs...
     [exec]
     [exec] /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/docs/solr-core/overview-summary.html
     [exec]   missing description: org.apache.solr.util.configuration.providers
     [exec]
     [exec] Missing javadocs were found!

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:810: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:101: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build.xml:669: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build.xml:685: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:2532: exec returned: 1

Total time: 136 minutes 0 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2092 - Still unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2092/

3 tests failed.
FAILED:  org.apache.solr.cloud.HealthCheckHandlerTest.testHealthCheckHandlerSolrJ

Error Message:
Error from server at http://127.0.0.1:37972/solr: Host Unavailable: Not connected to zk

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37972/solr: Host Unavailable: Not connected to zk
        at __randomizedtesting.SeedInfo.seed([6A41B6D4A9256BBB:C0B269C0D2C323BA]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
        at org.apache.solr.cloud.HealthCheckHandlerTest.testHealthCheckHandlerSolrJ(HealthCheckHandlerTest.java:78)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestSSLRandomization.testRandomizedSslAndClientAuth

Error Message:


Stack Trace:
java.lang.ExceptionInInitializerError
        at __randomizedtesting.SeedInfo.seed([6A41B6D4A9256BBB:E11507B30BBA2AC7]:0)
        at org.apache.solr.cloud.TestSSLRandomization.testRandomizedSslAndClientAuth(TestSSLRandomization.java:50)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Unable to initialize 'Default' SSLContext Algorithm, JVM is borked
        at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<clinit>(TestMiniSolrCloudClusterSSL.java:67)
        ... 40 more
Caused by: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: Default, provider: SunJSSE, class: sun.security.ssl.SSLContextImpl$DefaultSSLContext)
        at java.security.Provider$Service.newInstance(Provider.java:1617)
        at sun.security.jca.GetInstance.getInstance(GetInstance.java:236)
        at sun.security.jca.GetInstance.getInstance(GetInstance.java:164)
        at javax.net.ssl.SSLContext.getInstance(SSLContext.java:156)
        at javax.net.ssl.SSLContext.getDefault(SSLContext.java:96)
        at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.<clinit>(TestMiniSolrCloudClusterSSL.java:64)
        ... 40 more
Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
        at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:780)
        at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:56)
        at sun.security.provider.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:224)
        at sun.security.provider.JavaKeyStore$DualFormatJKS.engineLoad(JavaKeyStore.java:70)
        at java.security.KeyStore.load(KeyStore.java:1445)
        at sun.security.ssl.TrustManagerFactoryImpl.getCacertsKeyStore(TrustManagerFactoryImpl.java:226)
        at sun.security.ssl.SSLContextImpl$DefaultManagersHolder.getTrustManagers(SSLContextImpl.java:771)
        at sun.security.ssl.SSLContextImpl$DefaultManagersHolder.<clinit>(SSLContextImpl.java:748)
        at sun.security.ssl.SSLContextImpl$DefaultSSLContext.<init>(SSLContextImpl.java:913)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at java.security.Provider$Service.newInstance(Provider.java:1595)
        ... 45 more
Caused by: java.security.UnrecoverableKeyException: Password verification failed
        at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:778)
        ... 58 more


FAILED:  org.apache.solr.update.TestInPlaceUpdatesDistrib.test

Error Message:
This doc was supposed to have been deleted, but was: SolrDocument{id=0, title_s=title0, id_i=0, inplace_updatable_float=1.0, _version_=1577921057545781248, inplace_updatable_int_with_default=666, inplace_updatable_float_with_default=42.0}

Stack Trace:
java.lang.AssertionError: This doc was supposed to have been deleted, but was: SolrDocument{id=0, title_s=title0, id_i=0, inplace_updatable_float=1.0, _version_=1577921057545781248, inplace_updatable_int_with_default=666, inplace_updatable_float_with_default=42.0}
        at __randomizedtesting.SeedInfo.seed([6A41B6D4A9256BBB:E215890E07D90643]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at org.apache.solr.update.TestInPlaceUpdatesDistrib.reorderedDBQsSimpleTest(TestInPlaceUpdatesDistrib.java:247)
        at org.apache.solr.update.TestInPlaceUpdatesDistrib.test(TestInPlaceUpdatesDistrib.java:151)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 11563 lines...]
   [junit4] Suite: org.apache.solr.update.TestInPlaceUpdatesDistrib
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/init-core-data-001
   [junit4]   2> 1365484 WARN  (SUITE-TestInPlaceUpdatesDistrib-seed#[6A41B6D4A9256BBB]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 1365484 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[6A41B6D4A9256BBB]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1365485 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[6A41B6D4A9256BBB]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1365485 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[6A41B6D4A9256BBB]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1365485 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[6A41B6D4A9256BBB]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1365486 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[6A41B6D4A9256BBB]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 1365531 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[6A41B6D4A9256BBB]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1365650 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[6A41B6D4A9256BBB]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1365693 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[6A41B6D4A9256BBB]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=inplace-updates
   [junit4]   2> 1365744 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[6A41B6D4A9256BBB]-worker) [    ] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
   [junit4]   2> 1365855 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[6A41B6D4A9256BBB]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1365887 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[6A41B6D4A9256BBB]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1365887 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[6A41B6D4A9256BBB]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1365979 INFO  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1366160 INFO  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1366236 INFO  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=inplace-updates
   [junit4]   2> 1366239 INFO  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
   [junit4]   2> 1366239 INFO  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 1366240 INFO  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1366240 INFO  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1366240 INFO  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/init-core-data-001/]
   [junit4]   2> 1366260 WARN  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 1366438 INFO  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1366438 INFO  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1366439 INFO  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1366439 INFO  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1366440 INFO  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4a62fd09[collection1] main]
   [junit4]   2> 1366440 WARN  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 1366441 INFO  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1366441 INFO  (coreLoadExecutor-2204-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577920840559755264
   [junit4]   2> 1366443 INFO  (SUITE-TestInPlaceUpdatesDistrib-seed#[6A41B6D4A9256BBB]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 1366481 INFO  (searcherExecutor-2205-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4a62fd09[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1366483 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1366502 INFO  (Thread-1212) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1366502 INFO  (Thread-1212) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1366545 ERROR (Thread-1212) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1366614 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.ZkTestServer start zk server on port:42416
   [junit4]   2> 1366835 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1366836 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/schema-inplace-updates.xml to /configs/conf1/schema.xml
   [junit4]   2> 1366866 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1366867 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1366868 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1366869 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1366900 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1366901 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1366915 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1366916 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1366917 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1366930 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e5e6a4f5d0001, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1367108 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
   [junit4]   2> 1367574 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1367575 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4f9d0c09{/,null,AVAILABLE}
   [junit4]   2> 1367576 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1bc06c3a{SSL,[ssl, http/1.1]}{127.0.0.1:37893}
   [junit4]   2> 1367576 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.e.j.s.Server Started @1380972ms
   [junit4]   2> 1367576 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/tempDir-001/control/data, replicaType=NRT, hostContext=/, hostPort=37893, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/control-001/cores}
   [junit4]   2> 1367576 WARN  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.u.c.SSLConfigurations No keystore password found, using default.
   [junit4]   2> 1367576 WARN  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.u.c.SSLConfigurations No truststore password found, using default.
   [junit4]   2> 1367576 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.u.c.SSLConfigurations Setting javax.net.ssl.keyStorePassword
   [junit4]   2> 1367576 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.u.c.SSLConfigurations Setting javax.net.ssl.trustStorePassword
   [junit4]   2> 1367576 ERROR (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1367576 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1367576 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1367576 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1367576 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-07T22:16:20.099Z
   [junit4]   2> 1367659 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1367659 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/control-001/solr.xml
   [junit4]   2> 1367701 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1367703 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42416/solr
   [junit4]   2> 1368187 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:37893_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1368200 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:37893_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37893_
   [junit4]   2> 1368201 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:37893_    ] o.a.s.c.Overseer Overseer (id=98620052539768837-127.0.0.1:37893_-n_0000000000) starting
   [junit4]   2> 1368241 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:37893_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37893_
   [junit4]   2> 1368256 INFO  (zkCallback-923-thread-1-processing-n:127.0.0.1:37893_) [n:127.0.0.1:37893_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1368637 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:37893_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1368739 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:37893_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1368739 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:37893_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1368741 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:37893_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/control-001/cores
   [junit4]   2> 1369080 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1369080 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42416/solr ready
   [junit4]   2> 1369168 INFO  (qtp857001197-5408) [n:127.0.0.1:37893_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:37893_&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1369186 INFO  (OverseerThreadFactory-2215-thread-1-processing-n:127.0.0.1:37893_) [n:127.0.0.1:37893_    ] o.a.s.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 1369315 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1369327 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1369478 INFO  (zkCallback-923-thread-1-processing-n:127.0.0.1:37893_) [n:127.0.0.1:37893_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1370597 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1370651 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=inplace-updates
   [junit4]   2> 1370654 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
   [junit4]   2> 1370654 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 1370655 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1370655 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1370655 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 1370743 WARN  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 1371039 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1371039 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1371040 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1371040 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1371041 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4bff2cbb[control_collection_shard1_replica_n1] main]
   [junit4]   2> 1371054 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1371055 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1371055 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1371056 INFO  (searcherExecutor-2218-thread-1-processing-n:127.0.0.1:37893_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@4bff2cbb[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1371057 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577920845399982080
   [junit4]   2> 1371086 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1371086 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1371087 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:37893/control_collection_shard1_replica_n1/
   [junit4]   2> 1371087 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1371087 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:37893/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 1371087 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1371135 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:37893/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 1371269 INFO  (zkCallback-923-thread-1-processing-n:127.0.0.1:37893_) [n:127.0.0.1:37893_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1371307 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1371311 INFO  (qtp857001197-5411) [n:127.0.0.1:37893_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1996
   [junit4]   2> 1371335 INFO  (qtp857001197-5408) [n:127.0.0.1:37893_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1371480 INFO  (zkCallback-923-thread-2-processing-n:127.0.0.1:37893_) [n:127.0.0.1:37893_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1372335 INFO  (qtp857001197-5408) [n:127.0.0.1:37893_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:37893_&wt=javabin&version=2} status=0 QTime=3167
   [junit4]   2> 1372366 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e5e6a4f5d0006, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1372512 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1372513 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42416/solr ready
   [junit4]   2> 1372513 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1372654 INFO  (qtp857001197-5405) [n:127.0.0.1:37893_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1372686 INFO  (OverseerThreadFactory-2215-thread-2-processing-n:127.0.0.1:37893_) [n:127.0.0.1:37893_    ] o.a.s.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 1372688 INFO  (OverseerCollectionConfigSetProcessor-98620052539768837-127.0.0.1:37893_-n_0000000000) [n:127.0.0.1:37893_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1372690 WARN  (OverseerThreadFactory-2215-thread-2-processing-n:127.0.0.1:37893_) [n:127.0.0.1:37893_    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 1373069 INFO  (qtp857001197-5405) [n:127.0.0.1:37893_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1373069 INFO  (qtp857001197-5405) [n:127.0.0.1:37893_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=414
   [junit4]   2> 1373456 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/shard-1-001 of type TLOG
   [junit4]   2> 1373470 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1373532 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@51183a07{/,null,AVAILABLE}
   [junit4]   2> 1373532 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@207df967{SSL,[ssl, http/1.1]}{127.0.0.1:45872}
   [junit4]   2> 1373532 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.e.j.s.Server Started @1386929ms
   [junit4]   2> 1373533 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/tempDir-001/jetty1, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/, hostPort=45872, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/shard-1-001/cores}
   [junit4]   2> 1373533 WARN  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.u.c.SSLConfigurations No keystore password found, using default.
   [junit4]   2> 1373533 WARN  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.u.c.SSLConfigurations No truststore password found, using default.
   [junit4]   2> 1373533 ERROR (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1373533 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1373533 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1373533 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1373533 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-07T22:16:26.056Z
   [junit4]   2> 1373583 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1373583 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/shard-1-001/solr.xml
   [junit4]   2> 1373605 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1373622 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42416/solr
   [junit4]   2> 1373641 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:45872_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1373663 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:45872_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1373664 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:45872_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45872_
   [junit4]   2> 1373674 INFO  (zkCallback-935-thread-1-processing-n:127.0.0.1:45872_) [n:127.0.0.1:45872_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1373674 INFO  (zkCallback-930-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1373674 INFO  (zkCallback-923-thread-2-processing-n:127.0.0.1:37893_) [n:127.0.0.1:37893_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1374000 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:45872_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1374045 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:45872_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1374045 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:45872_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1374061 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:45872_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/shard-1-001/cores
   [junit4]   2> 1374393 INFO  (qtp857001197-5404) [n:127.0.0.1:37893_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:45872_&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1374437 INFO  (OverseerCollectionConfigSetProcessor-98620052539768837-127.0.0.1:37893_-n_0000000000) [n:127.0.0.1:37893_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1374439 INFO  (OverseerThreadFactory-2215-thread-3-processing-n:127.0.0.1:37893_) [n:127.0.0.1:37893_    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:45872_ for creating new replica
   [junit4]   2> 1374542 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 1374555 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1375694 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1375745 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.s.IndexSchema [collection1_shard1_replica_t21] Schema name=inplace-updates
   [junit4]   2> 1375748 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
   [junit4]   2> 1375748 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t21' using configuration from collection collection1, trusted=true
   [junit4]   2> 1375749 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_t21' (registry 'solr.core.collection1.shard1.replica_t21') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1375769 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1375769 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.SolrCore [[collection1_shard1_replica_t21] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/shard-1-001/cores/collection1_shard1_replica_t21], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/shard-1-001/cores/collection1_shard1_replica_t21/data/]
   [junit4]   2> 1375800 WARN  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 1376091 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1376091 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1376092 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1376092 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1376093 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.s.SolrIndexSearcher Opening [Searcher@4e693786[collection1_shard1_replica_t21] main]
   [junit4]   2> 1376094 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1376107 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1376107 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1376144 INFO  (searcherExecutor-2229-thread-1-processing-n:127.0.0.1:45872_ x:collection1_shard1_replica_t21 s:shard1 c:collection1) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.SolrCore [collection1_shard1_replica_t21] Registered new searcher Searcher@4e693786[collection1_shard1_replica_t21] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1376144 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577920850734088192
   [junit4]   2> 1376160 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1376160 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1376160 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:45872/collection1_shard1_replica_t21/
   [junit4]   2> 1376160 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1376160 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.SyncStrategy https://127.0.0.1:45872/collection1_shard1_replica_t21/ has no replicas
   [junit4]   2> 1376160 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1376161 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.ZkController collection1_shard1_replica_t21 stopping background replication from leader
   [junit4]   2> 1376162 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:45872/collection1_shard1_replica_t21/ shard1
   [junit4]   2> 1376319 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1376347 INFO  (qtp201892034-5451) [n:127.0.0.1:45872_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1804
   [junit4]   2> 1376444 INFO  (OverseerCollectionConfigSetProcessor-98620052539768837-127.0.0.1:37893_-n_0000000000) [n:127.0.0.1:37893_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1376444 INFO  (qtp857001197-5404) [n:127.0.0.1:37893_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:45872_&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2} status=0 QTime=2051
   [junit4]   2> 1376933 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/shard-2-001 of type TLOG
   [junit4]   2> 1376951 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1377086 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@319a24ad{/,null,AVAILABLE}
   [junit4]   2> 1377086 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2947047f{SSL,[ssl, http/1.1]}{127.0.0.1:39780}
   [junit4]   2> 1377086 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.e.j.s.Server Started @1390482ms
   [junit4]   2> 1377086 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/tempDir-001/jetty2, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/, hostPort=39780, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/shard-2-001/cores}
   [junit4]   2> 1377086 WARN  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.u.c.SSLConfigurations No keystore password found, using default.
   [junit4]   2> 1377106 WARN  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.u.c.SSLConfigurations No truststore password found, using default.
   [junit4]   2> 1377106 ERROR (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1377106 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1377106 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1377106 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1377106 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-07T22:16:29.629Z
   [junit4]   2> 1377183 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1377183 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/shard-2-001/solr.xml
   [junit4]   2> 1377247 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1377275 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42416/solr
   [junit4]   2> 1377422 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:39780_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1377452 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:39780_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1377454 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:39780_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39780_
   [junit4]   2> 1377454 INFO  (zkCallback-923-thread-2-processing-n:127.0.0.1:37893_) [n:127.0.0.1:37893_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1377475 INFO  (zkCallback-930-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1377488 INFO  (zkCallback-935-thread-1-processing-n:127.0.0.1:45872_) [n:127.0.0.1:45872_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1377549 INFO  (zkCallback-941-thread-1-processing-n:127.0.0.1:39780_) [n:127.0.0.1:39780_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1377878 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:39780_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1377911 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:39780_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1377912 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:39780_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1377913 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:39780_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/shard-2-001/cores
   [junit4]   2> 1378191 INFO  (qtp699753011-5480) [n:127.0.0.1:39780_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:39780_&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1378247 INFO  (OverseerThreadFactory-2215-thread-4-processing-n:127.0.0.1:37893_) [n:127.0.0.1:37893_    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:39780_ for creating new replica
   [junit4]   2> 1378287 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t23&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 1378287 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1379356 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1379460 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.s.IndexSchema [collection1_shard1_replica_t23] Schema name=inplace-updates
   [junit4]   2> 1379475 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
   [junit4]   2> 1379475 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t23' using configuration from collection collection1, trusted=true
   [junit4]   2> 1379475 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_t23' (registry 'solr.core.collection1.shard1.replica_t23') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1379476 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1379476 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.c.SolrCore [[collection1_shard1_replica_t23] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/shard-2-001/cores/collection1_shard1_replica_t23], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/shard-2-001/cores/collection1_shard1_replica_t23/data/]
   [junit4]   2> 1379488 WARN  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 1379582 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1379582 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1379583 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1379583 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1379600 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.s.SolrIndexSearcher Opening [Searcher@6e2f1d67[collection1_shard1_replica_t23] main]
   [junit4]   2> 1379601 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1379601 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1379602 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1379603 INFO  (searcherExecutor-2240-thread-1-processing-n:127.0.0.1:39780_ x:collection1_shard1_replica_t23 s:shard1 c:collection1) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.c.SolrCore [collection1_shard1_replica_t23] Registered new searcher Searcher@6e2f1d67[collection1_shard1_replica_t23] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1379603 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577920854361112576
   [junit4]   2> 1379606 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.c.ZkController Core needs to recover:collection1_shard1_replica_t23
   [junit4]   2> 1379624 INFO  (updateExecutor-938-thread-1-processing-n:127.0.0.1:39780_ x:collection1_shard1_replica_t23 s:shard1 c:collection1) [n:127.0.0.1:39780_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_t23] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 1379625 INFO  (qtp699753011-5478) [n:127.0.0.1:39780_ c:collection1 s:shard1  x:collection1_shard1_replica_t23] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t23&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1338
   [junit4]   2> 1379625 INFO  (recoveryExecutor-939-thread-1-processing-n:127.0.0.1:39780_ x:collection1_shard1_replica_t23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:39780_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_t23] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 1379625 INFO  (recoveryExecutor-939-thread-1-processing-n:127.0.0.1:39780_ x:collection1_shard1_replica_t23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:39780_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_t23] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 1379625 INFO  (recoveryExecutor-939-thread-1-processing-n:127.0.0.1:39780_ x:collection1_shard1_replica_t23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:39780_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_t23] o.a.s.c.ZkController collection1_shard1_replica_t23 stopping background replication from leader
   [junit4]   2> 1379625 INFO  (recoveryExecutor-939-thread-1-processing-n:127.0.0.1:39780_ x:collection1_shard1_replica_t23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:39780_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_t23] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard1_replica_t23]
   [junit4]   2> 1379625 INFO  (recoveryExecutor-939-thread-1-processing-n:127.0.0.1:39780_ x:collection1_shard1_replica_t23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:39780_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_t23] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 1379625 INFO  (recoveryExecutor-939-thread-1-processing-n:127.0.0.1:39780_ x:collection1_shard1_replica_t23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:39780_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_t23] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard1_replica_t23] as recovering, leader is [https://127.0.0.1:45872/collection1_shard1_replica_t21/] and I am [https://127.0.0.1:39780/collection1_shard1_replica_t23/]
   [junit4]   2> 1379626 INFO  (qtp699753011-5480) [n:127.0.0.1:39780_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:39780_&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2} status=0 QTime=1435
   [junit4]   2> 1379627 INFO  (recoveryExecutor-939-thread-1-processing-n:127.0.0.1:39780_ x:collection1_shard1_replica_t23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:39780_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_t23] o.a.s.c.RecoveryStrategy Sending prep recovery command to [https://127.0.0.1:45872]; [WaitForState: action=PREPRECOVERY&core=collection1_shard1_replica_t21&nodeName=127.0.0.1:39780_&coreNodeName=core_node24&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 1379647 INFO  (qtp201892034-5446) [n:127.0.0.1:45872_    ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node24, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
   [junit4]   2> 1379665 INFO  (qtp201892034-5446) [n:127.0.0.1:45872_    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_t21, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:39780_, coreNodeName=core_node24, onlyIfActiveCheckResult=false, nodeProps: core_node24:{"core":"collection1_shard1_replica_t23","base_url":"https://127.0.0.1:39780","node_name":"127.0.0.1:39780_","state":"down","type":"TLOG"}
   [junit4]   2> 1379901 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/shard-3-001 of type TLOG
   [junit4]   2> 1379903 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1379912 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@30cd441a{/,null,AVAILABLE}
   [junit4]   2> 1379913 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@59f5b54a{SSL,[ssl, http/1.1]}{127.0.0.1:33694}
   [junit4]   2> 1379913 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.e.j.s.Server Started @1393310ms
   [junit4]   2> 1379913 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/tempDir-001/jetty3, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/, hostPort=33694, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/shard-3-001/cores}
   [junit4]   2> 1379914 WARN  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.u.c.SSLConfigurations No keystore password found, using default.
   [junit4]   2> 1379914 WARN  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.u.c.SSLConfigurations No truststore password found, using default.
   [junit4]   2> 1379914 ERROR (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1379914 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1379914 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1379914 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1379914 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-07T22:16:32.437Z
   [junit4]   2> 1379926 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1379926 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/shard-3-001/solr.xml
   [junit4]   2> 1379958 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1379969 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42416/solr
   [junit4]   2> 1379987 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:33694_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1379988 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:33694_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1379989 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:33694_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33694_
   [junit4]   2> 1379989 INFO  (zkCallback-930-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1380034 INFO  (zkCallback-935-thread-1-processing-n:127.0.0.1:45872_) [n:127.0.0.1:45872_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1380035 INFO  (zkCallback-948-thread-2-processing-n:127.0.0.1:33694_) [n:127.0.0.1:33694_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1380035 INFO  (zkCallback-941-thread-1-processing-n:127.0.0.1:39780_) [n:127.0.0.1:39780_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1380058 INFO  (zkCallback-923-thread-1-processing-n:127.0.0.1:37893_) [n:127.0.0.1:37893_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1380270 INFO  (OverseerCollectionConfigSetProcessor-98620052539768837-127.0.0.1:37893_-n_0000000000) [n:127.0.0.1:37893_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1380336 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:33694_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1380361 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:33694_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1380361 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:33694_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4053a4
   [junit4]   2> 1380363 INFO  (TEST-TestInPlaceUpdatesDistrib.test-seed#[6A41B6D4A9256BBB]) [n:127.0.0.1:33694_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_6A41B6D4A9256BBB-001/shard-3-001/cores
   [junit4]   2> 1380521 INFO  (qtp699753011-5481) [n:127.0.0.1:39780_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:33694_&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1380539 INFO  (OverseerThreadFactory-2215-thread-5-processing-n:127.0.0.1:37893_) [n:127.0.0.1:37893_    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:33694_ for creating new replica
   [junit4]   2> 1380628 INFO  (qtp616137162-5513) [n:127.0.0.1:33694_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t25&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 1380628 INFO  (qtp616137162-5513) [n:127.0.0.1:33694_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1380672 INFO  (qtp201892034-5446) [n:127.0.0.1:45872_    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_t21, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:39780_, coreNodeName=core_node24, onlyIfActiveCheckResult=false, nodeProps: core_node24:{"core":"collection1_shard1_replica_t23","base_url":"https://127.0.0.1:39780","node_name":"127.0.0.1:39780_","state":"recovering","type":"TLOG"}
   [junit4]   2> 1380672 INFO  (qtp201892034-5446) [n:127.0.0.1:45872_    ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node24, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
   [junit4]   2> 1380672 INFO  (qtp201892034-5446) [n:127.0.0.1:45872_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:39780_&onlyIfLeaderActive=true&core=collection1_shard1_replica_t21&coreNodeName=core_node24&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1025
   [junit4]   2> 1381178 INFO  (recoveryExecutor-939-thread-1-processing-n:127.0.0.1:39780_ x:collection1_shard1_replica_t23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:39780_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_t23] o.a.s.c.RecoveryStrategy Starting Replication Recovery.
   [junit4]   2> 1381179 INFO  (recoveryExecutor-939-thread-1-processing-n:127.0.0.1:39780_ x:collection1_shard1_replica_t23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:39780_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_t23] o.a.s.c.RecoveryStrategy Attempting to replicate from [https://127.0.0.1:45872/collection1_shard1_replica_t21/].
   [junit4]   2> 1381182 INFO  (qtp201892034-5449) [n:127.0.0.1:45872_ c:collection1 s:shard1 r:core_node22 x:collection1_shard1_replica_t21] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577920856016814080,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1381182 INFO  (qtp201892034-5449) [n:127.0.0.1:45872_ c:collection1 s:shard1 r:core_node22 x:collection1_shard1_replica_t21] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1381183 INFO  (qtp201892034-5449) [n:127.0.0.1:45872_ c:collection1 s:shard1 r:core_node22 x:collection1_shard1_replica_t21] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1381183 INFO  (qtp201892034-5449) [n:127.0.0.1:45872_ c:collection1 s:shard1 r:core_node22 x:collection1_shard1_replica_t21] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_t21]  webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2}{commit=} 0 1
   [junit4]   2> 1381198 INFO  (qtp201892034-5446) [n:127.0.0.1:45872_ c:collection1 s:shard1 r:core_node22 x:collection1_shard1_replica_t21] o.a.s.c.S.Request [collection1_shard1_replica_t21]  webapp= path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 1381198 INFO  (recoveryExecutor-939-thread-1-processing-n:127.0.0.1:39780_ x:collection1_shard1_replica_t23 s:shard1 c:collection1 r:core_node

[...truncated too long message...]

Name = solr_43268, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2186fb59
   [junit4]   2> 3772709 INFO  (jetty-closer-1227-thread-3) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=805542610
   [junit4]   2> 3772758 INFO  (jetty-closer-1227-thread-3) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 3772758 INFO  (jetty-closer-1227-thread-3) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@44e46b03: rootName = solr_44287, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@774fce3c
   [junit4]   2> 3772770 INFO  (jetty-closer-1227-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 3772786 INFO  (jetty-closer-1227-thread-2) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3772790 INFO  (jetty-closer-1227-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 3772790 INFO  (jetty-closer-1227-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6694427: rootName = solr_42758, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@30ee68ab
   [junit4]   2> 3772793 INFO  (jetty-closer-1227-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 3772793 INFO  (jetty-closer-1227-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7a5ce01b: rootName = solr_42758, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2186fb59
   [junit4]   2> 3772803 INFO  (jetty-closer-1227-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4a623245{/solr,null,UNAVAILABLE}
   [junit4]   2> 3772807 INFO  (jetty-closer-1227-thread-3) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 3772807 INFO  (jetty-closer-1227-thread-3) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6b9c45b0: rootName = solr_44287, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@30ee68ab
   [junit4]   2> 3772810 INFO  (jetty-closer-1227-thread-3) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 3772810 INFO  (jetty-closer-1227-thread-3) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2ceb7d59: rootName = solr_44287, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2186fb59
   [junit4]   2> 3772814 INFO  (zkCallback-1245-thread-2-processing-n:127.0.0.1:42758_solr) [n:127.0.0.1:42758_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 3772822 INFO  (zkCallback-1238-thread-2-processing-n:127.0.0.1:44287_solr) [n:127.0.0.1:44287_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 3772823 INFO  (jetty-closer-1227-thread-3) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 3772823 INFO  (jetty-closer-1227-thread-3) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3772831 INFO  (jetty-closer-1227-thread-3) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@55dfecdb{/solr,null,UNAVAILABLE}
   [junit4]   2> 3772831 INFO  (zkCallback-1245-thread-1-processing-n:127.0.0.1:42758_solr) [n:127.0.0.1:42758_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 3772831 INFO  (jetty-closer-1227-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 3772831 INFO  (jetty-closer-1227-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3772831 INFO  (jetty-closer-1227-thread-1) [    ] o.a.s.c.Overseer Overseer (id=98620209948327945-127.0.0.1:42758_solr-n_0000000000) closing
   [junit4]   2> 3772832 INFO  (OverseerStateUpdate-98620209948327945-127.0.0.1:42758_solr-n_0000000000) [n:127.0.0.1:42758_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42758_solr
   [junit4]   2> 3772851 INFO  (jetty-closer-1227-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1e245c5{/solr,null,UNAVAILABLE}
   [junit4]   2> 3772928 ERROR (SUITE-TestSSLRandomization-seed#[6A41B6D4A9256BBB]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 3772944 INFO  (SUITE-TestSSLRandomization-seed#[6A41B6D4A9256BBB]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34739 34739
   [junit4]   2> 3773061 INFO  (Thread-2358) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34739 34739
   [junit4]   2> 3773062 WARN  (Thread-2358) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/aliases.json
   [junit4]   2> 3 /solr/security.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/clusterstate.json
   [junit4]   2> 3 /solr/clusterprops.json
   [junit4]   2> 2 /solr/overseer_elect/election/98620209948327945-127.0.0.1:42758_solr-n_0000000000
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/live_nodes
   [junit4]   2> 3 /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestSSLRandomization_6A41B6D4A9256BBB-001
   [junit4]   2> Sep 07, 2017 10:56:26 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1019, maxMBSortInHeap=5.3128656562158465, sim=RandomSimilarity(queryNorm=true): {}, locale=es, timezone=Pacific/Enderbury
   [junit4]   2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=161058368,total=489684992
   [junit4]   2> NOTE: All tests run in this JVM: [PeerSyncTest, TestConfig, SolrShardReporterTest, HLLUtilTest, LeaderElectionIntegrationTest, HttpPartitionTest, BasicDistributedZkTest, DocValuesMissingTest, TestPushWriter, AliasIntegrationTest, RankQueryTest, BJQParserTest, TestBinaryResponseWriter, TestLockTree, TestRealTimeGet, HttpSolrCallGetCoreTest, TestExtendedDismaxParser, TestSolrCoreParser, TestComponentsName, TestFunctionQuery, ExplicitHLLTest, TestRandomFlRTGCloud, PreAnalyzedUpdateProcessorTest, TestWordDelimiterFilterFactory, TestPivotHelperCode, TestSolrDeletionPolicy1, TestCorePropertiesReload, TestCollationFieldDocValues, SolrGangliaReporterTest, TestFastOutputStream, TestUseDocValuesAsStored2, MoveReplicaHDFSUlogDirTest, TestDistribIDF, TestMinMaxOnMultiValuedField, TestSizeLimitedDistributedMap, TestNumericRangeQuery64, TestJsonRequest, TestGeoJSONResponseWriter, BlockJoinFacetSimpleTest, TestGraphTermsQParserPlugin, DocumentAnalysisRequestHandlerTest, TestCopyFieldCollectionResource, TestStressUserVersions, DistributedFacetPivotSmallTest, SearchHandlerTest, TestNestedDocsSort, TestBadConfig, ZkStateWriterTest, TestHdfsBackupRestoreCore, TestRandomDVFaceting, TestManagedSchemaAPI, TestSubQueryTransformerCrossCore, ExternalFileFieldSortTest, BinaryUpdateRequestHandlerTest, SharedFSAutoReplicaFailoverTest, ConfigSetsAPITest, TestDocSet, ChaosMonkeyNothingIsSafeTest, TestHighlightDedupGrouping, TestEmbeddedSolrServerAdminHandler, SolrMetricManagerTest, HdfsRestartWhileUpdatingTest, SolrRequestParserTest, QueryResultKeyTest, RAMDirectoryFactoryTest, ZkNodePropsTest, DeleteNodeTest, JvmMetricsTest, TestIntervalFaceting, DefaultValueUpdateProcessorTest, ClassificationUpdateProcessorIntegrationTest, TestCollationField, DistributedQueryElevationComponentTest, SolrCLIZkUtilsTest, EchoParamsTest, TestDocumentBuilder, TestXmlQParserPlugin, DocValuesNotIndexedTest, TestFieldCache, NodeMutatorTest, PluginInfoTest, TestLMDirichletSimilarityFactory, TestConfigSetsAPIExclusivity, CachingDirectoryFactoryTest, XmlUpdateRequestHandlerTest, MetricsHandlerTest, HdfsChaosMonkeyNothingIsSafeTest, CursorMarkTest, TestLeaderElectionZkExpiry, TestPolicyCloud, StressHdfsTest, TestReplicaProperties, TestJettySolrRunner, TestDistributedSearch, SpellingQueryConverterTest, TestFieldTypeResource, DeleteStatusTest, CdcrVersionReplicationTest, SegmentsInfoRequestHandlerTest, TestDeleteCollectionOnDownNodes, TestExclusionRuleCollectionAccess, FieldAnalysisRequestHandlerTest, TestWriterPerf, TestNRTOpen, SolrCoreTest, TestBulkSchemaConcurrent, MBeansHandlerTest, SyncSliceTest, SuggesterWFSTTest, DirectoryFactoryTest, TestSchemaSimilarityResource, TestPHPSerializedResponseWriter, TestFileDictionaryLookup, TestLazyCores, TestSolrQueryResponse, TestLegacyField, TestSolr4Spatial, TestFaceting, TestSSLRandomization]
   [junit4] Completed [387/736 (3!)] on J0 in 6.65s, 4 tests, 1 error <<< FAILURES!

[...truncated 47366 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2093 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2093/

1 tests failed.
FAILED:  org.apache.solr.cloud.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete

Error Message:
Error from server at http://127.0.0.1:42061/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 </title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason: <pre>    Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/> </body> </html>

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:42061/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/>
</body>
</html>

        at __randomizedtesting.SeedInfo.seed([3820D0AE6F8DB799:9BDA7E0BE8655D3C]:0)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
        at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
        at org.apache.solr.cloud.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete(TestCollectionsAPIViaSolrCloudCluster.java:167)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42061/solr/testcollection_shard1_replica_n3: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n3/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n3/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/>
</body>
</html>

        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:591)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:516)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        ... 1 more




Build Log:
[...truncated 11949 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestCollectionsAPIViaSolrCloudCluster
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001/init-core-data-001
   [junit4]   2> 1113257 WARN  (SUITE-TestCollectionsAPIViaSolrCloudCluster-seed#[3820D0AE6F8DB799]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=21 numCloses=21
   [junit4]   2> 1113257 INFO  (SUITE-TestCollectionsAPIViaSolrCloudCluster-seed#[3820D0AE6F8DB799]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1113258 INFO  (SUITE-TestCollectionsAPIViaSolrCloudCluster-seed#[3820D0AE6F8DB799]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1113258 INFO  (SUITE-TestCollectionsAPIViaSolrCloudCluster-seed#[3820D0AE6F8DB799]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1113260 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 5 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001/tempDir-001
   [junit4]   2> 1113260 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1113272 INFO  (Thread-1298) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1113272 INFO  (Thread-1298) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1113321 ERROR (Thread-1298) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1113366 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.c.ZkTestServer start zk server on port:37840
   [junit4]   2> 1113438 INFO  (jetty-launcher-1044-thread-2) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1113447 INFO  (jetty-launcher-1044-thread-3) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1113457 INFO  (jetty-launcher-1044-thread-4) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1113476 INFO  (jetty-launcher-1044-thread-5) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1113535 INFO  (jetty-launcher-1044-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@23466a09{/solr,null,AVAILABLE}
   [junit4]   2> 1113539 INFO  (jetty-launcher-1044-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@ddfe9dd{HTTP/1.1,[http/1.1]}{127.0.0.1:42061}
   [junit4]   2> 1113539 INFO  (jetty-launcher-1044-thread-4) [    ] o.e.j.s.Server Started @1120272ms
   [junit4]   2> 1113539 INFO  (jetty-launcher-1044-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42061}
   [junit4]   2> 1113539 ERROR (jetty-launcher-1044-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1113539 INFO  (jetty-launcher-1044-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1113539 INFO  (jetty-launcher-1044-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1113539 INFO  (jetty-launcher-1044-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1113539 INFO  (jetty-launcher-1044-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-08T12:08:54.869Z
   [junit4]   2> 1113640 INFO  (jetty-launcher-1044-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@538c07ed{/solr,null,AVAILABLE}
   [junit4]   2> 1113641 INFO  (jetty-launcher-1044-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1deac8ad{HTTP/1.1,[http/1.1]}{127.0.0.1:43693}
   [junit4]   2> 1113641 INFO  (jetty-launcher-1044-thread-3) [    ] o.e.j.s.Server Started @1120374ms
   [junit4]   2> 1113641 INFO  (jetty-launcher-1044-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43693}
   [junit4]   2> 1113641 ERROR (jetty-launcher-1044-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1113641 INFO  (jetty-launcher-1044-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1113641 INFO  (jetty-launcher-1044-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1113641 INFO  (jetty-launcher-1044-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1113641 INFO  (jetty-launcher-1044-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-08T12:08:54.971Z
   [junit4]   2> 1113642 INFO  (jetty-launcher-1044-thread-1) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1113699 INFO  (jetty-launcher-1044-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1f7f7001{/solr,null,AVAILABLE}
   [junit4]   2> 1113699 INFO  (jetty-launcher-1044-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@69f74856{HTTP/1.1,[http/1.1]}{127.0.0.1:39196}
   [junit4]   2> 1113699 INFO  (jetty-launcher-1044-thread-1) [    ] o.e.j.s.Server Started @1120432ms
   [junit4]   2> 1113699 INFO  (jetty-launcher-1044-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39196}
   [junit4]   2> 1113699 ERROR (jetty-launcher-1044-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1113699 INFO  (jetty-launcher-1044-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1113699 INFO  (jetty-launcher-1044-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1113699 INFO  (jetty-launcher-1044-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1113699 INFO  (jetty-launcher-1044-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-08T12:08:55.029Z
   [junit4]   2> 1113733 INFO  (jetty-launcher-1044-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1113851 INFO  (jetty-launcher-1044-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@60ac537b{/solr,null,AVAILABLE}
   [junit4]   2> 1113851 INFO  (jetty-launcher-1044-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1a311f8f{HTTP/1.1,[http/1.1]}{127.0.0.1:58319}
   [junit4]   2> 1113852 INFO  (jetty-launcher-1044-thread-2) [    ] o.e.j.s.Server Started @1120585ms
   [junit4]   2> 1113852 INFO  (jetty-launcher-1044-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58319}
   [junit4]   2> 1113852 ERROR (jetty-launcher-1044-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1113852 INFO  (jetty-launcher-1044-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1113852 INFO  (jetty-launcher-1044-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1113852 INFO  (jetty-launcher-1044-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1113852 INFO  (jetty-launcher-1044-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-08T12:08:55.182Z
   [junit4]   2> 1113854 INFO  (jetty-launcher-1044-thread-5) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5c0c26f1{/solr,null,AVAILABLE}
   [junit4]   2> 1113854 INFO  (jetty-launcher-1044-thread-5) [    ] o.e.j.s.AbstractConnector Started ServerConnector@f8c750c{HTTP/1.1,[http/1.1]}{127.0.0.1:51982}
   [junit4]   2> 1113854 INFO  (jetty-launcher-1044-thread-5) [    ] o.e.j.s.Server Started @1120588ms
   [junit4]   2> 1113855 INFO  (jetty-launcher-1044-thread-5) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51982}
   [junit4]   2> 1113855 ERROR (jetty-launcher-1044-thread-5) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1113855 INFO  (jetty-launcher-1044-thread-5) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1113855 INFO  (jetty-launcher-1044-thread-5) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1113855 INFO  (jetty-launcher-1044-thread-5) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1113855 INFO  (jetty-launcher-1044-thread-5) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-08T12:08:55.185Z
   [junit4]   2> 1113862 INFO  (jetty-launcher-1044-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1113879 INFO  (jetty-launcher-1044-thread-5) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1113906 INFO  (jetty-launcher-1044-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37840/solr
   [junit4]   2> 1113915 INFO  (jetty-launcher-1044-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1113915 INFO  (jetty-launcher-1044-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1113925 INFO  (jetty-launcher-1044-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37840/solr
   [junit4]   2> 1113932 INFO  (jetty-launcher-1044-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37840/solr
   [junit4]   2> 1113960 INFO  (jetty-launcher-1044-thread-5) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37840/solr
   [junit4]   2> 1113934 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e616492600007, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1113994 INFO  (jetty-launcher-1044-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37840/solr
   [junit4]   2> 1113998 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e61649260000b, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1114054 INFO  (jetty-launcher-1044-thread-5) [n:127.0.0.1:51982_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1114055 INFO  (jetty-launcher-1044-thread-5) [n:127.0.0.1:51982_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51982_solr
   [junit4]   2> 1114055 INFO  (jetty-launcher-1044-thread-5) [n:127.0.0.1:51982_solr    ] o.a.s.c.Overseer Overseer (id=98623326429118477-127.0.0.1:51982_solr-n_0000000000) starting
   [junit4]   2> 1114070 INFO  (jetty-launcher-1044-thread-1) [n:127.0.0.1:39196_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1114070 INFO  (jetty-launcher-1044-thread-1) [n:127.0.0.1:39196_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39196_solr
   [junit4]   2> 1114090 INFO  (jetty-launcher-1044-thread-2) [n:127.0.0.1:58319_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1114091 INFO  (jetty-launcher-1044-thread-2) [n:127.0.0.1:58319_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1114093 INFO  (jetty-launcher-1044-thread-2) [n:127.0.0.1:58319_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58319_solr
   [junit4]   2> 1114101 INFO  (jetty-launcher-1044-thread-4) [n:127.0.0.1:42061_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1114103 INFO  (jetty-launcher-1044-thread-4) [n:127.0.0.1:42061_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1114104 INFO  (jetty-launcher-1044-thread-4) [n:127.0.0.1:42061_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42061_solr
   [junit4]   2> 1114110 INFO  (jetty-launcher-1044-thread-3) [n:127.0.0.1:43693_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1114111 INFO  (jetty-launcher-1044-thread-3) [n:127.0.0.1:43693_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1114112 INFO  (jetty-launcher-1044-thread-3) [n:127.0.0.1:43693_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43693_solr
   [junit4]   2> 1114191 INFO  (zkCallback-1070-thread-1-processing-n:127.0.0.1:51982_solr) [n:127.0.0.1:51982_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1114191 INFO  (zkCallback-1065-thread-1-processing-n:127.0.0.1:39196_solr) [n:127.0.0.1:39196_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1114216 INFO  (zkCallback-1070-thread-2-processing-n:127.0.0.1:51982_solr) [n:127.0.0.1:51982_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (4)
   [junit4]   2> 1114238 INFO  (zkCallback-1058-thread-1-processing-n:127.0.0.1:42061_solr) [n:127.0.0.1:42061_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
   [junit4]   2> 1114378 INFO  (zkCallback-1069-thread-1-processing-n:127.0.0.1:58319_solr) [n:127.0.0.1:58319_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (4)
   [junit4]   2> 1114395 INFO  (zkCallback-1072-thread-1-processing-n:127.0.0.1:43693_solr) [n:127.0.0.1:43693_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1114402 INFO  (jetty-launcher-1044-thread-3) [n:127.0.0.1:43693_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43693.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1114406 INFO  (jetty-launcher-1044-thread-5) [n:127.0.0.1:51982_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51982_solr
   [junit4]   2> 1114407 INFO  (zkCallback-1058-thread-1-processing-n:127.0.0.1:42061_solr) [n:127.0.0.1:42061_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1114408 INFO  (zkCallback-1065-thread-1-processing-n:127.0.0.1:39196_solr) [n:127.0.0.1:39196_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1114408 INFO  (zkCallback-1072-thread-1-processing-n:127.0.0.1:43693_solr) [n:127.0.0.1:43693_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1114408 INFO  (zkCallback-1069-thread-1-processing-n:127.0.0.1:58319_solr) [n:127.0.0.1:58319_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1114408 INFO  (zkCallback-1070-thread-2-processing-n:127.0.0.1:51982_solr) [n:127.0.0.1:51982_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1114439 INFO  (jetty-launcher-1044-thread-3) [n:127.0.0.1:43693_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43693.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1114439 INFO  (jetty-launcher-1044-thread-3) [n:127.0.0.1:43693_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43693.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1114440 INFO  (jetty-launcher-1044-thread-3) [n:127.0.0.1:43693_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001/tempDir-001/node3/.
   [junit4]   2> 1114686 INFO  (jetty-launcher-1044-thread-2) [n:127.0.0.1:58319_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58319.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1114691 INFO  (jetty-launcher-1044-thread-5) [n:127.0.0.1:51982_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51982.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1114697 INFO  (jetty-launcher-1044-thread-4) [n:127.0.0.1:42061_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42061.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1114709 INFO  (jetty-launcher-1044-thread-4) [n:127.0.0.1:42061_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42061.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1114709 INFO  (jetty-launcher-1044-thread-4) [n:127.0.0.1:42061_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42061.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1114722 INFO  (jetty-launcher-1044-thread-2) [n:127.0.0.1:58319_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58319.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1114722 INFO  (jetty-launcher-1044-thread-2) [n:127.0.0.1:58319_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58319.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1114724 INFO  (jetty-launcher-1044-thread-4) [n:127.0.0.1:42061_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001/tempDir-001/node4/.
   [junit4]   2> 1114724 INFO  (jetty-launcher-1044-thread-5) [n:127.0.0.1:51982_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51982.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1114724 INFO  (jetty-launcher-1044-thread-5) [n:127.0.0.1:51982_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51982.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1114725 INFO  (jetty-launcher-1044-thread-5) [n:127.0.0.1:51982_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001/tempDir-001/node5/.
   [junit4]   2> 1114730 INFO  (jetty-launcher-1044-thread-2) [n:127.0.0.1:58319_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001/tempDir-001/node2/.
   [junit4]   2> 1114765 INFO  (jetty-launcher-1044-thread-1) [n:127.0.0.1:39196_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39196.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1114825 INFO  (jetty-launcher-1044-thread-1) [n:127.0.0.1:39196_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39196.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1114825 INFO  (jetty-launcher-1044-thread-1) [n:127.0.0.1:39196_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39196.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1114826 INFO  (jetty-launcher-1044-thread-1) [n:127.0.0.1:39196_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001/tempDir-001/node1/.
   [junit4]   2> 1115072 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
   [junit4]   2> 1115072 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37840/solr ready
   [junit4]   2> 1115074 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCollectionCreateSearchDelete
   [junit4]   2> 1115077 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@1deac8ad{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1115077 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1921144817
   [junit4]   2> 1115077 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1115077 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@604a291b: rootName = solr_43693, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@6db8cbdf
   [junit4]   2> 1115083 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1115083 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@64383e0e: rootName = solr_43693, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@7c4014af
   [junit4]   2> 1115087 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1115087 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6afb6c0c: rootName = solr_43693, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1db81618
   [junit4]   2> 1115089 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1115090 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1115102 INFO  (zkCallback-1080-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 1115102 INFO  (zkCallback-1065-thread-1-processing-n:127.0.0.1:39196_solr) [n:127.0.0.1:39196_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 1115102 INFO  (zkCallback-1058-thread-1-processing-n:127.0.0.1:42061_solr) [n:127.0.0.1:42061_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 1115103 INFO  (zkCallback-1070-thread-2-processing-n:127.0.0.1:51982_solr) [n:127.0.0.1:51982_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 1115103 INFO  (zkCallback-1069-thread-1-processing-n:127.0.0.1:58319_solr) [n:127.0.0.1:58319_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 1116603 WARN  (zkCallback-1072-thread-1-processing-n:127.0.0.1:43693_solr) [n:127.0.0.1:43693_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1116605 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@538c07ed{/solr,null,UNAVAILABLE}
   [junit4]   2> 1116606 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1116607 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@48d7edac{/solr,null,AVAILABLE}
   [junit4]   2> 1116607 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@73d28272{HTTP/1.1,[http/1.1]}{127.0.0.1:60162}
   [junit4]   2> 1116607 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.e.j.s.Server Started @1123340ms
   [junit4]   2> 1116607 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=60162}
   [junit4]   2> 1116608 ERROR (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1116608 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1116608 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1116608 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1116608 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-08T12:08:57.938Z
   [junit4]   2> 1116618 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1116632 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37840/solr
   [junit4]   2> 1116641 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [n:127.0.0.1:60162_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1116642 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [n:127.0.0.1:60162_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1116644 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [n:127.0.0.1:60162_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60162_solr
   [junit4]   2> 1116645 INFO  (zkCallback-1080-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1116645 INFO  (zkCallback-1058-thread-1-processing-n:127.0.0.1:42061_solr) [n:127.0.0.1:42061_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1116645 INFO  (zkCallback-1070-thread-2-processing-n:127.0.0.1:51982_solr) [n:127.0.0.1:51982_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1116646 INFO  (zkCallback-1065-thread-1-processing-n:127.0.0.1:39196_solr) [n:127.0.0.1:39196_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1116646 INFO  (zkCallback-1069-thread-1-processing-n:127.0.0.1:58319_solr) [n:127.0.0.1:58319_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1116657 INFO  (zkCallback-1085-thread-2-processing-n:127.0.0.1:60162_solr) [n:127.0.0.1:60162_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1116695 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [n:127.0.0.1:60162_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60162.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1116705 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [n:127.0.0.1:60162_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60162.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1116705 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [n:127.0.0.1:60162_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60162.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1116706 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [n:127.0.0.1:60162_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001/tempDir-001/node6/.
   [junit4]   2> 1116778 INFO  (qtp2030287044-6133) [n:127.0.0.1:58319_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=1&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1116780 INFO  (OverseerThreadFactory-2715-thread-1-processing-n:127.0.0.1:51982_solr) [n:127.0.0.1:51982_solr    ] o.a.s.c.CreateCollectionCmd Create collection testcollection
   [junit4]   2> 1116891 INFO  (OverseerStateUpdate-98623326429118477-127.0.0.1:51982_solr-n_0000000000) [n:127.0.0.1:51982_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:58319/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1116901 INFO  (OverseerStateUpdate-98623326429118477-127.0.0.1:51982_solr-n_0000000000) [n:127.0.0.1:51982_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n3",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42061/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1116904 INFO  (OverseerStateUpdate-98623326429118477-127.0.0.1:51982_solr-n_0000000000) [n:127.0.0.1:51982_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n5",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:60162/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1116909 INFO  (OverseerStateUpdate-98623326429118477-127.0.0.1:51982_solr-n_0000000000) [n:127.0.0.1:51982_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n7",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:39196/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1117013 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node2&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
   [junit4]   2> 1117021 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1117033 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node4&name=testcollection_shard1_replica_n3&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
   [junit4]   2> 1117033 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1117042 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node6&name=testcollection_shard2_replica_n5&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
   [junit4]   2> 1117042 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1117042 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node8&name=testcollection_shard2_replica_n7&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
   [junit4]   2> 1117043 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1117153 INFO  (zkCallback-1085-thread-2-processing-n:127.0.0.1:60162_solr) [n:127.0.0.1:60162_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1117153 INFO  (zkCallback-1058-thread-1-processing-n:127.0.0.1:42061_solr) [n:127.0.0.1:42061_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1117153 INFO  (zkCallback-1085-thread-1-processing-n:127.0.0.1:60162_solr) [n:127.0.0.1:60162_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1117153 INFO  (zkCallback-1065-thread-1-processing-n:127.0.0.1:39196_solr) [n:127.0.0.1:39196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1117154 INFO  (zkCallback-1065-thread-2-processing-n:127.0.0.1:39196_solr) [n:127.0.0.1:39196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1117154 INFO  (zkCallback-1069-thread-2-processing-n:127.0.0.1:58319_solr) [n:127.0.0.1:58319_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1117153 INFO  (zkCallback-1069-thread-1-processing-n:127.0.0.1:58319_solr) [n:127.0.0.1:58319_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1117169 INFO  (zkCallback-1058-thread-2-processing-n:127.0.0.1:42061_solr) [n:127.0.0.1:42061_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1118034 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1118057 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1118059 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1118077 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.s.IndexSchema [testcollection_shard2_replica_n5] Schema name=minimal
   [junit4]   2> 1118080 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1118080 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard2_replica_n5' using configuration from collection testcollection, trusted=true
   [junit4]   2> 1118080 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60162.solr.core.testcollection.shard2.replica_n5' (registry 'solr.core.testcollection.shard2.replica_n5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1118080 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1118080 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.SolrCore [[testcollection_shard2_replica_n5] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001/tempDir-001/node6/testcollection_shard2_replica_n5], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001/tempDir-001/node6/./testcollection_shard2_replica_n5/data/]
   [junit4]   2> 1118084 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema [testcollection_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 1118086 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1118086 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1118086 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n1' using configuration from collection testcollection, trusted=true
   [junit4]   2> 1118087 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58319.solr.core.testcollection.shard1.replica_n1' (registry 'solr.core.testcollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1118087 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1118087 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [[testcollection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001/tempDir-001/node2/testcollection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001/tempDir-001/node2/./testcollection_shard1_replica_n1/data/]
   [junit4]   2> 1118091 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.s.IndexSchema [testcollection_shard1_replica_n3] Schema name=minimal
   [junit4]   2> 1118093 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1118093 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n3' using configuration from collection testcollection, trusted=true
   [junit4]   2> 1118093 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42061.solr.core.testcollection.shard1.replica_n3' (registry 'solr.core.testcollection.shard1.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1118093 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1118094 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.SolrCore [[testcollection_shard1_replica_n3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001/tempDir-001/node4/testcollection_shard1_replica_n3], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001/tempDir-001/node4/./testcollection_shard1_replica_n3/data/]
   [junit4]   2> 1118096 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.s.IndexSchema [testcollection_shard2_replica_n7] Schema name=minimal
   [junit4]   2> 1118098 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1118098 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard2_replica_n7' using configuration from collection testcollection, trusted=true
   [junit4]   2> 1118098 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39196.solr.core.testcollection.shard2.replica_n7' (registry 'solr.core.testcollection.shard2.replica_n7') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1d8890ea
   [junit4]   2> 1118106 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1118106 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.c.SolrCore [[testcollection_shard2_replica_n7] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001/tempDir-001/node1/testcollection_shard2_replica_n7], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001/tempDir-001/node1/./testcollection_shard2_replica_n7/data/]
   [junit4]   2> 1118250 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1118250 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1118251 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1118251 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1118266 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@57273ae9[testcollection_shard1_replica_n3] main]
   [junit4]   2> 1118267 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 1118267 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 1118268 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1118278 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1118278 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1118282 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1118282 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1118283 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1118283 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1118283 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1118283 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1118284 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577973228098289664
   [junit4]   2> 1118288 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 1118290 INFO  (searcherExecutor-2735-thread-1-processing-n:127.0.0.1:42061_solr x:testcollection_shard1_replica_n3 s:shard1 c:testcollection r:core_node4) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.SolrCore [testcollection_shard1_replica_n3] Registered new searcher Searcher@57273ae9[testcollection_shard1_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1118292 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.s.SolrIndexSearcher Opening [Searcher@65c18621[testcollection_shard2_replica_n5] main]
   [junit4]   2> 1118293 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 1118293 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 1118293 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1118294 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577973228108775424
   [junit4]   2> 1118298 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 1118298 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.s.SolrIndexSearcher Opening [Searcher@12899505[testcollection_shard2_replica_n7] main]
   [junit4]   2> 1118300 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 1118300 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 1118301 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1118301 INFO  (searcherExecutor-2733-thread-1-processing-n:127.0.0.1:60162_solr x:testcollection_shard2_replica_n5 s:shard2 c:testcollection r:core_node6) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.SolrCore [testcollection_shard2_replica_n5] Registered new searcher Searcher@65c18621[testcollection_shard2_replica_n5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1118307 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577973228122406912
   [junit4]   2> 1118307 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1118307 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1118308 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1118308 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1118310 INFO  (searcherExecutor-2736-thread-1-processing-n:127.0.0.1:39196_solr x:testcollection_shard2_replica_n7 s:shard2 c:testcollection r:core_node8) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.c.SolrCore [testcollection_shard2_replica_n7] Registered new searcher Searcher@12899505[testcollection_shard2_replica_n7] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1118310 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4c8c6b0e[testcollection_shard1_replica_n1] main]
   [junit4]   2> 1118322 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 1118323 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 1118323 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1118323 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577973228139184128
   [junit4]   2> 1118324 INFO  (searcherExecutor-2734-thread-1-processing-n:127.0.0.1:58319_solr x:testcollection_shard1_replica_n1 s:shard1 c:testcollection r:core_node2) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [testcollection_shard1_replica_n1] Registered new searcher Searcher@4c8c6b0e[testcollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1118400 INFO  (zkCallback-1058-thread-2-processing-n:127.0.0.1:42061_solr) [n:127.0.0.1:42061_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1118400 INFO  (zkCallback-1058-thread-1-processing-n:127.0.0.1:42061_solr) [n:127.0.0.1:42061_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1118400 INFO  (zkCallback-1069-thread-2-processing-n:127.0.0.1:58319_solr) [n:127.0.0.1:58319_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1118400 INFO  (zkCallback-1069-thread-1-processing-n:127.0.0.1:58319_solr) [n:127.0.0.1:58319_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1118400 INFO  (zkCallback-1065-thread-1-processing-n:127.0.0.1:39196_solr) [n:127.0.0.1:39196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1118400 INFO  (zkCallback-1065-thread-2-processing-n:127.0.0.1:39196_solr) [n:127.0.0.1:39196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1118400 INFO  (zkCallback-1085-thread-1-processing-n:127.0.0.1:60162_solr) [n:127.0.0.1:60162_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1118400 INFO  (zkCallback-1085-thread-2-processing-n:127.0.0.1:60162_solr) [n:127.0.0.1:60162_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1118790 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1118790 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1118790 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42061/solr/testcollection_shard1_replica_n3/
   [junit4]   2> 1118791 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=testcollection_shard1_replica_n3 url=http://127.0.0.1:42061/solr START replicas=[http://127.0.0.1:58319/solr/testcollection_shard1_replica_n1/] nUpdates=100
   [junit4]   2> 1118799 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1118799 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1118799 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60162/solr/testcollection_shard2_replica_n5/
   [junit4]   2> 1118799 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.PeerSync PeerSync: core=testcollection_shard2_replica_n5 url=http://127.0.0.1:60162/solr START replicas=[http://127.0.0.1:39196/solr/testcollection_shard2_replica_n7/] nUpdates=100
   [junit4]   2> 1118822 INFO  (qtp2030287044-6135) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.c.S.Request [testcollection_shard1_replica_n1]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=11
   [junit4]   2> 1118838 INFO  (qtp671366815-6120) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.c.S.Request [testcollection_shard2_replica_n7]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=15
   [junit4]   2> 1119180 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.PeerSync PeerSync: core=testcollection_shard2_replica_n5 url=http://127.0.0.1:60162/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 1119180 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 1119180 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 1119180 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1119180 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=testcollection_shard1_replica_n3 url=http://127.0.0.1:42061/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 1119180 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 1119180 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 1119181 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1119185 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42061/solr/testcollection_shard1_replica_n3/ shard1
   [junit4]   2> 1119185 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60162/solr/testcollection_shard2_replica_n5/ shard2
   [junit4]   2> 1119288 INFO  (zkCallback-1069-thread-2-processing-n:127.0.0.1:58319_solr) [n:127.0.0.1:58319_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1119288 INFO  (zkCallback-1069-thread-1-processing-n:127.0.0.1:58319_solr) [n:127.0.0.1:58319_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1119288 INFO  (zkCallback-1065-thread-1-processing-n:127.0.0.1:39196_solr) [n:127.0.0.1:39196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1119288 INFO  (zkCallback-1065-thread-2-processing-n:127.0.0.1:39196_solr) [n:127.0.0.1:39196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1119288 INFO  (zkCallback-1058-thread-2-processing-n:127.0.0.1:42061_solr) [n:127.0.0.1:42061_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1119288 INFO  (zkCallback-1058-thread-1-processing-n:127.0.0.1:42061_solr) [n:127.0.0.1:42061_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1119289 INFO  (zkCallback-1085-thread-1-processing-n:127.0.0.1:60162_solr) [n:127.0.0.1:60162_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1119289 INFO  (zkCallback-1085-thread-2-processing-n:127.0.0.1:60162_solr) [n:127.0.0.1:60162_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1119315 INFO  (qtp671366815-6122) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node8&name=testcollection_shard2_replica_n7&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=2273
   [junit4]   2> 1119333 INFO  (qtp2030287044-6131) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node2&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=2319
   [junit4]   2> 1119336 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1119336 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1119337 INFO  (qtp2108138983-6101) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node4&name=testcollection_shard1_replica_n3&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=2304
   [junit4]   2> 1119337 INFO  (qtp713467715-6224) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node6&name=testcollection_shard2_replica_n5&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=2295
   [junit4]   2> 1119340 INFO  (qtp2030287044-6133) [n:127.0.0.1:58319_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1119441 INFO  (zkCallback-1069-thread-2-processing-n:127.0.0.1:58319_solr) [n:127.0.0.1:58319_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1119441 INFO  (zkCallback-1069-thread-1-processing-n:127.0.0.1:58319_solr) [n:127.0.0.1:58319_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1119441 INFO  (zkCallback-1065-thread-1-processing-n:127.0.0.1:39196_solr) [n:127.0.0.1:39196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1119441 INFO  (zkCallback-1065-thread-2-processing-n:127.0.0.1:39196_solr) [n:127.0.0.1:39196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1119441 INFO  (zkCallback-1058-thread-2-processing-n:127.0.0.1:42061_solr) [n:127.0.0.1:42061_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1119441 INFO  (zkCallback-1058-thread-1-processing-n:127.0.0.1:42061_solr) [n:127.0.0.1:42061_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1119441 INFO  (zkCallback-1085-thread-2-processing-n:127.0.0.1:60162_solr) [n:127.0.0.1:60162_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1119441 INFO  (zkCallback-1085-thread-1-processing-n:127.0.0.1:60162_solr) [n:127.0.0.1:60162_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1120340 INFO  (qtp2030287044-6133) [n:127.0.0.1:58319_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=1&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=3561
   [junit4]   2> 1120341 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: testcollection failOnTimeout:true timeout (sec):330
   [junit4]   1> -
   [junit4]   1> replica:core_node2 rstate:active live:true
   [junit4]   1> replica:core_node4 rstate:active live:true
   [junit4]   1> replica:core_node6 rstate:active live:true
   [junit4]   1> replica:core_node8 rstate:active live:true
   [junit4]   1> no one is recoverying
   [junit4]   2> 1120341 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: testcollection
   [junit4]   2> 1120360 INFO  (qtp2030287044-6136) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:42061/solr/testcollection_shard1_replica_n3/&wt=javabin&version=2}{add=[1 (1577973230259404800)]} 0 11
   [junit4]   2> 1120360 INFO  (qtp2108138983-6099) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n3]  webapp=/solr path=/update params={_stateVer_=testcollection:5&wt=javabin&version=2}{add=[1 (1577973230259404800)]} 0 16
   [junit4]   2> 1120363 INFO  (qtp2108138983-6098) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577973230278279168,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1120363 INFO  (qtp2108138983-6098) [n:127.0.0.1:42061_solr c:testcollection s:shard1 r:core_node4 x:testcollection_shard1_replica_n3] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@667b9618 commitCommandVersion:1577973230278279168
   [junit4]   2> 1120363 INFO  (qtp2030287044-6137) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577973230278279168,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1120363 INFO  (qtp2030287044-6137) [n:127.0.0.1:58319_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1ab2f3ba commitCommandVersion:1577973230278279168
   [junit4]   2> 1120377 INFO  (qtp713467715-6222) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577973230292959232,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1120378 INFO  (qtp713467715-6222) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1120379 INFO  (qtp713467715-6222) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1120379 INFO  (qtp713467715-6222) [n:127.0.0.1:60162_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n5] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard2_replica_n5]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:42061/solr/testcollection_shard1_replica_n3/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 1
   [junit4]   2> 1120380 INFO  (qtp671366815-6124) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577973230296104960,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1120381 INFO  (qtp671366815-6124) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1120383 INFO  (qtp671366815-6124) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1120383 INFO  (qtp671366815-6124) [n:127.0.0.1:39196_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard2_rep

[...truncated too long message...]

BLE}
   [junit4]   2> 1157245 INFO  (jetty-closer-1157-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1157237 INFO  (jetty-closer-1157-thread-5) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1157246 INFO  (jetty-closer-1157-thread-5) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@78c9152d: rootName = solr_33166, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@7c4014af
   [junit4]   2> 1157234 INFO  (zkCallback-1172-thread-1-processing-n:127.0.0.1:44360_solr) [n:127.0.0.1:44360_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 1157234 INFO  (zkCallback-1185-thread-2-processing-n:127.0.0.1:33166_solr) [n:127.0.0.1:33166_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 1157234 INFO  (zkCallback-1184-thread-2-processing-n:127.0.0.1:47216_solr) [n:127.0.0.1:47216_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 1157233 INFO  (jetty-closer-1157-thread-2) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1157249 INFO  (jetty-closer-1157-thread-5) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1157249 INFO  (jetty-closer-1157-thread-5) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@77e15fbf: rootName = solr_33166, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1db81618
   [junit4]   2> 1157250 INFO  (jetty-closer-1157-thread-2) [    ] o.a.s.c.Overseer Overseer (id=98623329152663561-127.0.0.1:44360_solr-n_0000000000) closing
   [junit4]   2> 1157250 INFO  (OverseerStateUpdate-98623329152663561-127.0.0.1:44360_solr-n_0000000000) [n:127.0.0.1:44360_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:44360_solr
   [junit4]   2> 1157251 WARN  (zkCallback-1183-thread-2-processing-n:127.0.0.1:53583_solr) [n:127.0.0.1:53583_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1157251 INFO  (jetty-closer-1157-thread-5) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1157251 INFO  (jetty-closer-1157-thread-5) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1157253 INFO  (zkCallback-1185-thread-3-processing-n:127.0.0.1:33166_solr) [n:127.0.0.1:33166_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (2)
   [junit4]   2> 1157253 INFO  (jetty-closer-1157-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@331b4ff4{/solr,null,UNAVAILABLE}
   [junit4]   2> 1157254 INFO  (zkCallback-1172-thread-3-processing-n:127.0.0.1:44360_solr) [n:127.0.0.1:44360_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (2)
   [junit4]   2> 1157259 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e616534b60009, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1157259 WARN  (zkCallback-1185-thread-3-processing-n:127.0.0.1:33166_solr) [n:127.0.0.1:33166_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1157260 INFO  (jetty-closer-1157-thread-5) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7faa12dd{/solr,null,UNAVAILABLE}
   [junit4]   2> 1157260 WARN  (zkCallback-1172-thread-3-processing-n:127.0.0.1:44360_solr) [n:127.0.0.1:44360_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1157260 INFO  (jetty-closer-1157-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5fc05ae2{/solr,null,UNAVAILABLE}
   [junit4]   2> 1158746 WARN  (zkCallback-1184-thread-2-processing-n:127.0.0.1:47216_solr) [n:127.0.0.1:47216_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1158747 INFO  (jetty-closer-1157-thread-3) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4f21f4a0{/solr,null,UNAVAILABLE}
   [junit4]   2> 1158748 ERROR (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateWithoutCoresThenDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1158748 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateWithoutCoresThenDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35037 35037
   [junit4]   2> 1158839 INFO  (Thread-1460) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35037 35037
   [junit4]   2> 1158840 WARN  (Thread-1460) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 6 /solr/aliases.json
   [junit4]   2> 6 /solr/clusterprops.json
   [junit4]   2> 5 /solr/security.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 6 /solr/clusterstate.json
   [junit4]   2> 3 /solr/overseer_elect/election/98623329152663561-127.0.0.1:44360_solr-n_0000000000
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 6 /solr/live_nodes
   [junit4]   2> 6 /solr/collections
   [junit4]   2>
   [junit4]   2> 1158840 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateWithoutCoresThenDelete-seed#[3820D0AE6F8DB799]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testCollectionCreateWithoutCoresThenDelete
   [junit4]   2> 1158841 INFO  (SUITE-TestCollectionsAPIViaSolrCloudCluster-seed#[3820D0AE6F8DB799]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35037 35037
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_3820D0AE6F8DB799-001
   [junit4]   2> Sep 08, 2017 12:09:40 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=15006, maxDocsPerChunk=121, blockSize=611), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=15006, blockSize=611)), sim=RandomSimilarity(queryNorm=true): {}, locale=nn-NO, timezone=America/Antigua
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=130387376,total=523239424
   [junit4]   2> NOTE: All tests run in this JVM: [JSONWriterTest, DocValuesTest, HighlighterTest, TestPartialUpdateDeduplication, TermVectorComponentTest, TestInfoStreamLogging, TestCharFilters, TestStressRecovery, MetricUtilsTest, TestScoreJoinQPNoScore, TestBackupRepositoryFactory, TestNumericTokenStream, SpellCheckComponentTest, SaslZkACLProviderTest, DataDrivenBlockJoinTest, SliceStateTest, ShardRoutingCustomTest, TestReloadAndDeleteDocs, TestSearchPerf, CollectionsAPISolrJTest, TestSolrCloudWithDelegationTokens, DocValuesMultiTest, SampleTest, TestRestManager, ImplicitSnitchTest, HdfsBasicDistributedZk2Test, TestTrieFacet, SortSpecParsingTest, CdcrReplicationHandlerTest, SharedFSAutoReplicaFailoverUtilsTest, ForceLeaderTest, TestPullReplica, TestNoOpRegenerator, SchemaVersionSpecificBehaviorTest, BlockJoinFacetDistribTest, TestPostingsSolrHighlighter, TestFreeTextSuggestions, DistributedExpandComponentTest, IndexSchemaRuntimeFieldTest, TestBlobHandler, DistributedIntervalFacetingTest, TestCloudSearcherWarming, HdfsChaosMonkeySafeLeaderTest, ZkCLITest, TestDocBasedVersionConstraints, TestSubQueryTransformerDistrib, DisMaxRequestHandlerTest, ConfigureRecoveryStrategyTest, ExitableDirectoryReaderTest, TestLegacyFieldReuse, HdfsLockFactoryTest, OverseerTaskQueueTest, SubstringBytesRefFilterTest, TestUnifiedSolrHighlighter, TestSchemalessBufferedUpdates, TestSegmentSorting, TestOrdValues, MoveReplicaTest, VersionInfoTest, SOLR749Test, TestSolrConfigHandlerConcurrent, TestXIncludeConfig, AnalyticsMergeStrategyTest, TestRawResponseWriter, CollectionStateFormat2Test, TestFieldResource, DOMUtilTest, ZkFailoverTest, SolrCmdDistributorTest, TestAnalyzeInfixSuggestions, TestReplicationHandlerBackup, UpdateRequestProcessorFactoryTest, SystemInfoHandlerTest, TestRTGBase, V2StandaloneTest, SolrMetricReporterTest, CloudMLTQParserTest, TestCoreContainer, TestNumericTerms32, TestLeaderInitiatedRecoveryThread, ConnectionReuseTest, LargeFieldTest, TestExpandComponent, TestFastLRUCache, TestCollectionsAPIViaSolrCloudCluster]
   [junit4] Completed [252/732 (1!)] on J1 in 46.63s, 3 tests, 1 error <<< FAILURES!

[...truncated 47820 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2094 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2094/

2 tests failed.
FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
KeeperErrorCode = Session expired for /clusterstate.json

Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
        at __randomizedtesting.SeedInfo.seed([A47251CD2A8E11C7:42E5650D130CE8A6]:0)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
        at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
        at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
        at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
        at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
        at org.apache.solr.common.cloud.ZkStateReader.refreshLegacyClusterState(ZkStateReader.java:542)
        at org.apache.solr.common.cloud.ZkStateReader.forceUpdateCollection(ZkStateReader.java:310)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.updateMappingsFromZk(AbstractFullDistribZkTestBase.java:674)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.updateMappingsFromZk(AbstractFullDistribZkTestBase.java:669)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:464)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:334)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestSegmentSorting.testSegmentTerminateEarly

Error Message:
responseHeader.segmentTerminatedEarly missing/false in {zkConnected=true,segmentTerminatedEarly=false,status=0,QTime=16}

Stack Trace:
java.lang.AssertionError: responseHeader.segmentTerminatedEarly missing/false in {zkConnected=true,segmentTerminatedEarly=false,status=0,QTime=16}
        at __randomizedtesting.SeedInfo.seed([A47251CD2A8E11C7:74D49660F6DC85AD]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.SegmentTerminateEarlyTestState.queryTimestampDescendingSegmentTerminateEarlyYes(SegmentTerminateEarlyTestState.java:150)
        at org.apache.solr.cloud.TestSegmentSorting.testSegmentTerminateEarly(TestSegmentSorting.java:109)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 11759 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestSegmentSorting
   [junit4]   2> 1326727 INFO  (SUITE-TestSegmentSorting-seed#[A47251CD2A8E11C7]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestSegmentSorting_A47251CD2A8E11C7-001/init-core-data-001
   [junit4]   2> 1326744 WARN  (SUITE-TestSegmentSorting-seed#[A47251CD2A8E11C7]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=60 numCloses=60
   [junit4]   2> 1326744 INFO  (SUITE-TestSegmentSorting-seed#[A47251CD2A8E11C7]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1326745 INFO  (SUITE-TestSegmentSorting-seed#[A47251CD2A8E11C7]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1326746 INFO  (SUITE-TestSegmentSorting-seed#[A47251CD2A8E11C7]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 5 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestSegmentSorting_A47251CD2A8E11C7-001/tempDir-001
   [junit4]   2> 1326746 INFO  (SUITE-TestSegmentSorting-seed#[A47251CD2A8E11C7]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1326746 INFO  (Thread-1098) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1326746 INFO  (Thread-1098) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1326747 ERROR (Thread-1098) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1326884 INFO  (SUITE-TestSegmentSorting-seed#[A47251CD2A8E11C7]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:39356
   [junit4]   2> 1326956 INFO  (jetty-launcher-809-thread-1) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1326956 INFO  (jetty-launcher-809-thread-2) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1326980 INFO  (jetty-launcher-809-thread-3) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1326984 INFO  (jetty-launcher-809-thread-4) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1327019 INFO  (jetty-launcher-809-thread-5) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1327067 INFO  (jetty-launcher-809-thread-5) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@445f6dd1{/solr,null,AVAILABLE}
   [junit4]   2> 1327068 INFO  (jetty-launcher-809-thread-5) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3b9d4adb{HTTP/1.1,[http/1.1]}{127.0.0.1:40014}
   [junit4]   2> 1327068 INFO  (jetty-launcher-809-thread-5) [    ] o.e.j.s.Server Started @1339616ms
   [junit4]   2> 1327068 INFO  (jetty-launcher-809-thread-5) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40014}
   [junit4]   2> 1327068 ERROR (jetty-launcher-809-thread-5) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1327068 INFO  (jetty-launcher-809-thread-5) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1327068 INFO  (jetty-launcher-809-thread-5) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1327068 INFO  (jetty-launcher-809-thread-5) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1327081 INFO  (jetty-launcher-809-thread-5) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-09T09:49:40.545Z
   [junit4]   2> 1327122 INFO  (jetty-launcher-809-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@74bccb2d{/solr,null,AVAILABLE}
   [junit4]   2> 1327122 INFO  (jetty-launcher-809-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3a906203{HTTP/1.1,[http/1.1]}{127.0.0.1:42387}
   [junit4]   2> 1327122 INFO  (jetty-launcher-809-thread-4) [    ] o.e.j.s.Server Started @1339670ms
   [junit4]   2> 1327122 INFO  (jetty-launcher-809-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42387}
   [junit4]   2> 1327122 ERROR (jetty-launcher-809-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1327122 INFO  (jetty-launcher-809-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1327122 INFO  (jetty-launcher-809-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1327122 INFO  (jetty-launcher-809-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1327122 INFO  (jetty-launcher-809-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-09T09:49:40.599Z
   [junit4]   2> 1327149 INFO  (jetty-launcher-809-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@88368ad{/solr,null,AVAILABLE}
   [junit4]   2> 1327149 INFO  (jetty-launcher-809-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@20829b70{HTTP/1.1,[http/1.1]}{127.0.0.1:39605}
   [junit4]   2> 1327149 INFO  (jetty-launcher-809-thread-1) [    ] o.e.j.s.Server Started @1339697ms
   [junit4]   2> 1327149 INFO  (jetty-launcher-809-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39605}
   [junit4]   2> 1327149 ERROR (jetty-launcher-809-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1327150 INFO  (jetty-launcher-809-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1327150 INFO  (jetty-launcher-809-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1327150 INFO  (jetty-launcher-809-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1327150 INFO  (jetty-launcher-809-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-09T09:49:40.627Z
   [junit4]   2> 1327163 INFO  (jetty-launcher-809-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@441f27fa{/solr,null,AVAILABLE}
   [junit4]   2> 1327163 INFO  (jetty-launcher-809-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@aacdceb{HTTP/1.1,[http/1.1]}{127.0.0.1:40777}
   [junit4]   2> 1327163 INFO  (jetty-launcher-809-thread-2) [    ] o.e.j.s.Server Started @1339711ms
   [junit4]   2> 1327163 INFO  (jetty-launcher-809-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40777}
   [junit4]   2> 1327163 ERROR (jetty-launcher-809-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1327163 INFO  (jetty-launcher-809-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1327163 INFO  (jetty-launcher-809-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1327163 INFO  (jetty-launcher-809-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1327163 INFO  (jetty-launcher-809-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-09T09:49:40.640Z
   [junit4]   2> 1327179 INFO  (jetty-launcher-809-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1327268 INFO  (jetty-launcher-809-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@728a99db{/solr,null,AVAILABLE}
   [junit4]   2> 1327268 INFO  (jetty-launcher-809-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@690c7bbc{HTTP/1.1,[http/1.1]}{127.0.0.1:34537}
   [junit4]   2> 1327268 INFO  (jetty-launcher-809-thread-3) [    ] o.e.j.s.Server Started @1339816ms
   [junit4]   2> 1327268 INFO  (jetty-launcher-809-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34537}
   [junit4]   2> 1327269 ERROR (jetty-launcher-809-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1327269 INFO  (jetty-launcher-809-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1327269 INFO  (jetty-launcher-809-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1327269 INFO  (jetty-launcher-809-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1327269 INFO  (jetty-launcher-809-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-09-09T09:49:40.746Z
   [junit4]   2> 1327296 INFO  (jetty-launcher-809-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1327366 INFO  (jetty-launcher-809-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1327399 INFO  (jetty-launcher-809-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39356/solr
   [junit4]   2> 1327401 INFO  (jetty-launcher-809-thread-5) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1327402 INFO  (jetty-launcher-809-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1327416 INFO  (jetty-launcher-809-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39356/solr
   [junit4]   2> 1327435 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e660b74000006, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1327525 INFO  (jetty-launcher-809-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39356/solr
   [junit4]   2> 1327690 INFO  (jetty-launcher-809-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39356/solr
   [junit4]   2> 1327788 INFO  (jetty-launcher-809-thread-5) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39356/solr
   [junit4]   2> 1328264 INFO  (jetty-launcher-809-thread-1) [n:127.0.0.1:39605_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1328265 INFO  (jetty-launcher-809-thread-1) [n:127.0.0.1:39605_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39605_solr
   [junit4]   2> 1328266 INFO  (jetty-launcher-809-thread-1) [n:127.0.0.1:39605_solr    ] o.a.s.c.Overseer Overseer (id=98628441225560073-127.0.0.1:39605_solr-n_0000000000) starting
   [junit4]   2> 1328267 INFO  (jetty-launcher-809-thread-3) [n:127.0.0.1:34537_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1328279 INFO  (jetty-launcher-809-thread-2) [n:127.0.0.1:40777_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1328280 INFO  (jetty-launcher-809-thread-2) [n:127.0.0.1:40777_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40777_solr
   [junit4]   2> 1328377 INFO  (zkCallback-837-thread-1-processing-n:127.0.0.1:40777_solr) [n:127.0.0.1:40777_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1328424 INFO  (jetty-launcher-809-thread-5) [n:127.0.0.1:40014_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1328425 INFO  (jetty-launcher-809-thread-5) [n:127.0.0.1:40014_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1328426 INFO  (zkCallback-827-thread-1-processing-n:127.0.0.1:34537_solr) [n:127.0.0.1:34537_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1328426 INFO  (jetty-launcher-809-thread-3) [n:127.0.0.1:34537_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34537_solr
   [junit4]   2> 1328427 INFO  (zkCallback-837-thread-1-processing-n:127.0.0.1:40777_solr) [n:127.0.0.1:40777_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1328428 INFO  (zkCallback-827-thread-1-processing-n:127.0.0.1:34537_solr) [n:127.0.0.1:34537_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1328428 INFO  (jetty-launcher-809-thread-5) [n:127.0.0.1:40014_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40014_solr
   [junit4]   2> 1328489 INFO  (zkCallback-826-thread-1-processing-n:127.0.0.1:39605_solr) [n:127.0.0.1:39605_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1328506 INFO  (zkCallback-838-thread-1-processing-n:127.0.0.1:40014_solr) [n:127.0.0.1:40014_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
   [junit4]   2> 1328519 INFO  (zkCallback-826-thread-2-processing-n:127.0.0.1:39605_solr) [n:127.0.0.1:39605_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
   [junit4]   2> 1328527 INFO  (zkCallback-827-thread-1-processing-n:127.0.0.1:34537_solr) [n:127.0.0.1:34537_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1328527 INFO  (zkCallback-837-thread-1-processing-n:127.0.0.1:40777_solr) [n:127.0.0.1:40777_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1328787 INFO  (jetty-launcher-809-thread-1) [n:127.0.0.1:39605_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39605_solr
   [junit4]   2> 1328889 INFO  (zkCallback-827-thread-1-processing-n:127.0.0.1:34537_solr) [n:127.0.0.1:34537_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1328890 INFO  (zkCallback-837-thread-1-processing-n:127.0.0.1:40777_solr) [n:127.0.0.1:40777_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1328897 INFO  (zkCallback-838-thread-1-processing-n:127.0.0.1:40014_solr) [n:127.0.0.1:40014_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1328913 INFO  (zkCallback-826-thread-2-processing-n:127.0.0.1:39605_solr) [n:127.0.0.1:39605_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1329325 INFO  (jetty-launcher-809-thread-5) [n:127.0.0.1:40014_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40014.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb3243b
   [junit4]   2> 1329454 INFO  (jetty-launcher-809-thread-5) [n:127.0.0.1:40014_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40014.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb3243b
   [junit4]   2> 1329574 INFO  (jetty-launcher-809-thread-3) [n:127.0.0.1:34537_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34537.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb3243b
   [junit4]   2> 1329972 INFO  (jetty-launcher-809-thread-3) [n:127.0.0.1:34537_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34537.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb3243b
   [junit4]   2> 1329972 INFO  (jetty-launcher-809-thread-3) [n:127.0.0.1:34537_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34537.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb3243b
   [junit4]   2> 1329973 INFO  (jetty-launcher-809-thread-3) [n:127.0.0.1:34537_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestSegmentSorting_A47251CD2A8E11C7-001/tempDir-001/node3/.
   [junit4]   2> 1330210 INFO  (jetty-launcher-809-thread-5) [n:127.0.0.1:40014_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40014.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb3243b
   [junit4]   2> 1330211 INFO  (jetty-launcher-809-thread-5) [n:127.0.0.1:40014_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestSegmentSorting_A47251CD2A8E11C7-001/tempDir-001/node5/.
   [junit4]   2> 1330228 INFO  (jetty-launcher-809-thread-2) [n:127.0.0.1:40777_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40777.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb3243b
   [junit4]   2> 1330301 INFO  (jetty-launcher-809-thread-2) [n:127.0.0.1:40777_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40777.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb3243b
   [junit4]   2> 1330301 INFO  (jetty-launcher-809-thread-2) [n:127.0.0.1:40777_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40777.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb3243b
   [junit4]   2> 1330302 INFO  (jetty-launcher-809-thread-2) [n:127.0.0.1:40777_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestSegmentSorting_A47251CD2A8E11C7-001/tempDir-001/node2/.
   [junit4]   2> 1330333 INFO  (jetty-launcher-809-thread-1) [n:127.0.0.1:39605_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39605.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb3243b
   [junit4]   2> 1330364 INFO  (jetty-launcher-809-thread-1) [n:127.0.0.1:39605_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39605.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb3243b
   [junit4]   2> 1330364 INFO  (jetty-launcher-809-thread-1) [n:127.0.0.1:39605_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39605.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb3243b
   [junit4]   2> 1330365 INFO  (jetty-launcher-809-thread-1) [n:127.0.0.1:39605_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestSegmentSorting_A47251CD2A8E11C7-001/tempDir-001/node1/.
   [junit4]   2> 1330560 INFO  (jetty-launcher-809-thread-4) [n:127.0.0.1:42387_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1330562 INFO  (jetty-launcher-809-thread-4) [n:127.0.0.1:42387_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1330575 INFO  (jetty-launcher-809-thread-4) [n:127.0.0.1:42387_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42387_solr
   [junit4]   2> 1330576 INFO  (zkCallback-826-thread-2-processing-n:127.0.0.1:39605_solr) [n:127.0.0.1:39605_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1330576 INFO  (zkCallback-837-thread-1-processing-n:127.0.0.1:40777_solr) [n:127.0.0.1:40777_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1330603 INFO  (zkCallback-838-thread-1-processing-n:127.0.0.1:40014_solr) [n:127.0.0.1:40014_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1330604 INFO  (zkCallback-827-thread-1-processing-n:127.0.0.1:34537_solr) [n:127.0.0.1:34537_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1330668 INFO  (zkCallback-832-thread-1-processing-n:127.0.0.1:42387_solr) [n:127.0.0.1:42387_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1330902 INFO  (jetty-launcher-809-thread-4) [n:127.0.0.1:42387_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42387.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb3243b
   [junit4]   2> 1330934 INFO  (jetty-launcher-809-thread-4) [n:127.0.0.1:42387_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42387.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb3243b
   [junit4]   2> 1330934 INFO  (jetty-launcher-809-thread-4) [n:127.0.0.1:42387_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42387.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb3243b
   [junit4]   2> 1330935 INFO  (jetty-launcher-809-thread-4) [n:127.0.0.1:42387_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestSegmentSorting_A47251CD2A8E11C7-001/tempDir-001/node4/.
   [junit4]   2> 1334386 INFO  (SUITE-TestSegmentSorting-seed#[A47251CD2A8E11C7]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
   [junit4]   2> 1334387 INFO  (SUITE-TestSegmentSorting-seed#[A47251CD2A8E11C7]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39356/solr ready
   [junit4]   2> 1336099 INFO  (TEST-TestSegmentSorting.testSegmentTerminateEarly-seed#[A47251CD2A8E11C7]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSegmentTerminateEarly
   [junit4]   2> 1336149 INFO  (qtp751624077-5520) [n:127.0.0.1:34537_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=f72d6cf4-1389-4ccf-b339-c735a93fbdc0&replicationFactor=2&property.config=solrconfig-sortingmergepolicyfactory.xml&collection.configName=class+org.apache.solr.cloud.TestSegmentSorting_configSet&name=testSegmentTerminateEarly&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1336169 INFO  (qtp751624077-5520) [n:127.0.0.1:34537_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=f72d6cf4-1389-4ccf-b339-c735a93fbdc0&replicationFactor=2&property.config=solrconfig-sortingmergepolicyfactory.xml&collection.configName=class+org.apache.solr.cloud.TestSegmentSorting_configSet&name=testSegmentTerminateEarly&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=19
   [junit4]   2> 1336170 INFO  (qtp751624077-5521) [n:127.0.0.1:34537_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=f72d6cf4-1389-4ccf-b339-c735a93fbdc0&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1336171 INFO  (qtp751624077-5521) [n:127.0.0.1:34537_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=f72d6cf4-1389-4ccf-b339-c735a93fbdc0&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1336185 INFO  (OverseerThreadFactory-1929-thread-1-processing-n:127.0.0.1:39605_solr) [n:127.0.0.1:39605_solr    ] o.a.s.c.CreateCollectionCmd Create collection testSegmentTerminateEarly
   [junit4]   2> 1336312 INFO  (OverseerStateUpdate-98628441225560073-127.0.0.1:39605_solr-n_0000000000) [n:127.0.0.1:39605_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testSegmentTerminateEarly",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testSegmentTerminateEarly_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42387/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1336332 INFO  (OverseerStateUpdate-98628441225560073-127.0.0.1:39605_solr-n_0000000000) [n:127.0.0.1:39605_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testSegmentTerminateEarly",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testSegmentTerminateEarly_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:40777/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1336334 INFO  (OverseerStateUpdate-98628441225560073-127.0.0.1:39605_solr-n_0000000000) [n:127.0.0.1:39605_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testSegmentTerminateEarly",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testSegmentTerminateEarly_shard2_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:40014/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1336345 INFO  (OverseerStateUpdate-98628441225560073-127.0.0.1:39605_solr-n_0000000000) [n:127.0.0.1:39605_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testSegmentTerminateEarly",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testSegmentTerminateEarly_shard2_replica_n6",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:39605/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1336581 INFO  (qtp1150821310-5484) [n:127.0.0.1:42387_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=class+org.apache.solr.cloud.TestSegmentSorting_configSet&newCollection=true&collection=testSegmentTerminateEarly&version=2&replicaType=NRT&async=f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899066962702&property.config=solrconfig-sortingmergepolicyfactory.xml&coreNodeName=core_node3&name=testSegmentTerminateEarly_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=0
   [junit4]   2> 1336608 INFO  (parallelCoreAdminExecutor-1938-thread-1-processing-n:127.0.0.1:42387_solr f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899066962702 CREATE) [n:127.0.0.1:42387_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=class+org.apache.solr.cloud.TestSegmentSorting_configSet&newCollection=true&collection=testSegmentTerminateEarly&version=2&replicaType=NRT&async=f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899066962702&property.config=solrconfig-sortingmergepolicyfactory.xml&coreNodeName=core_node3&name=testSegmentTerminateEarly_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 1336608 INFO  (qtp742521253-5505) [n:127.0.0.1:40777_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=class+org.apache.solr.cloud.TestSegmentSorting_configSet&newCollection=true&collection=testSegmentTerminateEarly&version=2&replicaType=NRT&async=f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899070053230&property.config=solrconfig-sortingmergepolicyfactory.xml&coreNodeName=core_node5&name=testSegmentTerminateEarly_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=0
   [junit4]   2> 1336609 INFO  (qtp1483620124-5475) [n:127.0.0.1:40014_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=class+org.apache.solr.cloud.TestSegmentSorting_configSet&newCollection=true&collection=testSegmentTerminateEarly&version=2&replicaType=NRT&async=f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899089572668&property.config=solrconfig-sortingmergepolicyfactory.xml&coreNodeName=core_node7&name=testSegmentTerminateEarly_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=0
   [junit4]   2> 1336610 INFO  (parallelCoreAdminExecutor-1932-thread-1-processing-n:127.0.0.1:40777_solr f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899070053230 CREATE) [n:127.0.0.1:40777_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=class+org.apache.solr.cloud.TestSegmentSorting_configSet&newCollection=true&collection=testSegmentTerminateEarly&version=2&replicaType=NRT&async=f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899070053230&property.config=solrconfig-sortingmergepolicyfactory.xml&coreNodeName=core_node5&name=testSegmentTerminateEarly_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 1336610 INFO  (parallelCoreAdminExecutor-1930-thread-1-processing-n:127.0.0.1:40014_solr f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899089572668 CREATE) [n:127.0.0.1:40014_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=class+org.apache.solr.cloud.TestSegmentSorting_configSet&newCollection=true&collection=testSegmentTerminateEarly&version=2&replicaType=NRT&async=f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899089572668&property.config=solrconfig-sortingmergepolicyfactory.xml&coreNodeName=core_node7&name=testSegmentTerminateEarly_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 1336609 INFO  (parallelCoreAdminExecutor-1935-thread-1-processing-n:127.0.0.1:39605_solr f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899100789961 CREATE) [n:127.0.0.1:39605_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=class+org.apache.solr.cloud.TestSegmentSorting_configSet&newCollection=true&collection=testSegmentTerminateEarly&version=2&replicaType=NRT&async=f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899100789961&property.config=solrconfig-sortingmergepolicyfactory.xml&coreNodeName=core_node8&name=testSegmentTerminateEarly_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 1336609 INFO  (parallelCoreAdminExecutor-1938-thread-1-processing-n:127.0.0.1:42387_solr f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899066962702 CREATE) [n:127.0.0.1:42387_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1336609 INFO  (qtp1894219689-5497) [n:127.0.0.1:39605_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=class+org.apache.solr.cloud.TestSegmentSorting_configSet&newCollection=true&collection=testSegmentTerminateEarly&version=2&replicaType=NRT&async=f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899100789961&property.config=solrconfig-sortingmergepolicyfactory.xml&coreNodeName=core_node8&name=testSegmentTerminateEarly_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=1
   [junit4]   2> 1336610 INFO  (parallelCoreAdminExecutor-1935-thread-1-processing-n:127.0.0.1:39605_solr f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899100789961 CREATE) [n:127.0.0.1:39605_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1336611 INFO  (qtp1894219689-5498) [n:127.0.0.1:39605_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899100789961
   [junit4]   2> 1336611 INFO  (qtp1894219689-5498) [n:127.0.0.1:39605_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899100789961&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1336611 INFO  (parallelCoreAdminExecutor-1932-thread-1-processing-n:127.0.0.1:40777_solr f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899070053230 CREATE) [n:127.0.0.1:40777_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1336619 INFO  (parallelCoreAdminExecutor-1930-thread-1-processing-n:127.0.0.1:40014_solr f72d6cf4-1389-4ccf-b339-c735a93fbdc05059899089572668 CREATE) [n:127.0.0.1:40014_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1336751 INFO  (zkCallback-832-thread-1-processing-n:127.0.0.1:42387_solr) [n:127.0.0.1:42387_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testSegmentTerminateEarly/state.json] for collection [testSegmentTerminateEarly] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1336751 INFO  (zkCallback-838-thread-1-processing-n:127.0.0.1:40014_