[jira] [Commented] (SOLR-13031) Harden ChaosMonkeySafeLeaderWithPullReplicasTest

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-13031) Harden ChaosMonkeySafeLeaderWithPullReplicasTest

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/SOLR-13031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16710728#comment-16710728 ]

Hoss Man commented on SOLR-13031:
---------------------------------


I can't find any examples of jenkins fialures of this test due to unreleased objects, but here's an example i found locally while beasting.

I should note that this seed does not reliably reproduce these leaked objects on my machine...

{noformat}
  [beaster]   2> 85175 INFO  (SUITE-ChaosMonkeySafeLeaderWithPullReplicasTest-seed#[65C44CE967C0F88A]-worker) [    ] o.a.s.SolrTestCaseJ4 Waiting for all tracked resources to be released
  [beaster]   2> 95177 INFO  (SUITE-ChaosMonkeySafeLeaderWithPullReplicasTest-seed#[65C44CE967C0F88A]-worker) [    ] o.a.s.SolrTestCaseJ4 Waiting for all tracked resources to be released
  [beaster]   2> 105179 INFO  (SUITE-ChaosMonkeySafeLeaderWithPullReplicasTest-seed#[65C44CE967C0F88A]-worker) [    ] o.a.s.SolrTestCaseJ4 Waiting for all tracked resources to be released
  [beaster]   2> Reference Handler:
  [beaster]   2>     java.lang.Object.wait(Native Method)
  [beaster]   2>     java.lang.Object.wait(Object.java:502)
  [beaster]   2>     java.lang.ref.Reference.tryHandlePending(Reference.java:191)
  [beaster]   2>     java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
  [beaster]   2>
  [beaster]   2> Signal Dispatcher:
  [beaster]   2>
  [beaster]   2> SUITE-ChaosMonkeySafeLeaderWithPullReplicasTest-seed#[65C44CE967C0F88A]-worker:
  [beaster]   2>     java.lang.Thread.dumpThreads(Native Method)
  [beaster]   2>     java.lang.Thread.getAllStackTraces(Thread.java:1610)
  [beaster]   2>     org.apache.solr.SolrTestCaseJ4.clearObjectTrackerAndCheckEmpty(SolrTestCaseJ4.java:388)
  [beaster]   2>     org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:316)
  [beaster]   2>     [...sun.*, java.lang.reflect.*, com.carrotsearch.randomizedtesting.*]
  [beaster]   2>     org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
  [beaster]   2>     [...com.carrotsearch.randomizedtesting.*]
  [beaster]   2>     org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
  [beaster]   2>     [...com.carrotsearch.randomizedtesting.*]
  [beaster]   2>     org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  [beaster]   2>     org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
  [beaster]   2>     org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
  [beaster]   2>     org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
  [beaster]   2>     [...com.carrotsearch.randomizedtesting.*]
  [beaster]   2>     java.lang.Thread.run(Thread.java:748)
  [beaster]   2>
  [beaster]   2> main:
  [beaster]   2>     java.lang.Object.wait(Native Method)
  [beaster]   2>     java.lang.Thread.join(Thread.java:1252)
  [beaster]   2>     java.lang.Thread.join(Thread.java:1326)
  [beaster]   2>     [...com.carrotsearch.randomizedtesting.*]
  [beaster]   2>     com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:269)
  [beaster]   2>     com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:394)
  [beaster]   2>     com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
  [beaster]   2>
  [beaster]   2> JUnit4-serializer-daemon:
  [beaster]   2>     java.lang.Thread.sleep(Native Method)
  [beaster]   2>     com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
  [beaster]   2>
  [beaster]   2> Finalizer:
  [beaster]   2>     java.lang.Object.wait(Native Method)
  [beaster]   2>     java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
  [beaster]   2>     java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
  [beaster]   2>     java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
  [beaster]   2>
  [beaster]   2> SUITE-ChaosMonkeySafeLeaderWithPullReplicasTest-seed#[65C44CE967C0F88A]:
  [beaster]   2>     java.lang.Object.wait(Native Method)
  [beaster]   2>     java.lang.Thread.join(Thread.java:1260)
  [beaster]   2>     [...com.carrotsearch.randomizedtesting.*]
  [beaster]   2>
  [beaster]   2> 115181 INFO  (SUITE-ChaosMonkeySafeLeaderWithPullReplicasTest-seed#[65C44CE967C0F88A]-worker) [    ] o.a.s.SolrTestCaseJ4 Waiting for all tracked resources to be released
  [beaster]   2> Reference Handler:
  [beaster]   2>     java.lang.Object.wait(Native Method)
  [beaster]   2>     java.lang.Object.wait(Object.java:502)
  [beaster]   2>     java.lang.ref.Reference.tryHandlePending(Reference.java:191)
  [beaster]   2>     java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
  [beaster]   2>
  [beaster]   2> Signal Dispatcher:
  [beaster]   2>
  [beaster]   2> SUITE-ChaosMonkeySafeLeaderWithPullReplicasTest-seed#[65C44CE967C0F88A]-worker:
  [beaster]   2>     java.lang.Thread.dumpThreads(Native Method)
  [beaster]   2>     java.lang.Thread.getAllStackTraces(Thread.java:1610)
  [beaster]   2>     org.apache.solr.SolrTestCaseJ4.clearObjectTrackerAndCheckEmpty(SolrTestCaseJ4.java:388)
  [beaster]   2>     org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:316)
  [beaster]   2>     [...sun.*, java.lang.reflect.*, com.carrotsearch.randomizedtesting.*]
  [beaster]   2>     org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
  [beaster]   2>     [...com.carrotsearch.randomizedtesting.*]
  [beaster]   2>     org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
  [beaster]   2>     [...com.carrotsearch.randomizedtesting.*]
  [beaster]   2>     org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  [beaster]   2>     org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
  [beaster]   2>     org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
  [beaster]   2>     org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
  [beaster]   2>     [...com.carrotsearch.randomizedtesting.*]
  [beaster]   2>     java.lang.Thread.run(Thread.java:748)
  [beaster]   2>
  [beaster]   2> main:
  [beaster]   2>     java.lang.Object.wait(Native Method)
  [beaster]   2>     java.lang.Thread.join(Thread.java:1252)
  [beaster]   2>     java.lang.Thread.join(Thread.java:1326)
  [beaster]   2>     [...com.carrotsearch.randomizedtesting.*]
  [beaster]   2>     com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:269)
  [beaster]   2>     com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:394)
  [beaster]   2>     com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
  [beaster]   2>
  [beaster]   2> JUnit4-serializer-daemon:
  [beaster]   2>     java.lang.Thread.sleep(Native Method)
  [beaster]   2>     com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
  [beaster]   2>
  [beaster]   2> Finalizer:
  [beaster]   2>     java.lang.Object.wait(Native Method)
  [beaster]   2>     java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
  [beaster]   2>     java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
  [beaster]   2>     java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
  [beaster]   2>
  [beaster]   2> SUITE-ChaosMonkeySafeLeaderWithPullReplicasTest-seed#[65C44CE967C0F88A]:
  [beaster]   2>     java.lang.Object.wait(Native Method)
  [beaster]   2>     java.lang.Thread.join(Thread.java:1260)
  [beaster]   2>     [...com.carrotsearch.randomizedtesting.*]
  [beaster]   2>
  [beaster]   2> 116182 INFO  (SUITE-ChaosMonkeySafeLeaderWithPullReplicasTest-seed#[65C44CE967C0F88A]-worker) [    ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
  [beaster]   2> NOTE: test params are: codec=Asserting(Lucene80): {rnd_b=PostingsFormat(name=Asserting), a_t=PostingsFormat(name=LuceneFixedGap), _root_=BlockTreeOrds(blocksize=128), id=PostingsFormat(name=Asserting)}, docValues:{rnd_b=DocValuesFormat(name=Asserting), _version_=DocValuesFormat(name=Lucene70), a_t=DocValuesFormat(name=Lucene70), _root_=DocValuesFormat(name=Lucene70), a_i=DocValuesFormat(name=Asserting), id=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=873, maxMBSortInHeap=7.2630995406398124, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@2ff266fa), locale=en-IN, timezone=Asia/Yangon
  [beaster]   2> NOTE: Linux 3.19.0-84-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=217328272,total=358612992
  [beaster]   2> NOTE: All tests run in this JVM: [ChaosMonkeySafeLeaderWithPullReplicasTest]
  [beaster]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeySafeLeaderWithPullReplicasTest -Dtests.seed=65C44CE967C0F88A -Dtests.multiplier=3 -Dtests.nightly=true -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-IN -Dtests.timezone=Asia/Yangon -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
  [beaster] [15:51:51.607] ERROR   0.00s | ChaosMonkeySafeLeaderWithPullReplicasTest (suite) <<<
  [beaster]    > Throwable #1: java.lang.AssertionError: ObjectTracker found 6 object(s) that were not released!!! [MMapDirectory, InternalHttpClient, MMapDirectory, SolrCore, MMapDirectory, MMapDirectory]
  [beaster]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
  [beaster]    > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  [beaster]    > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
  [beaster]    > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:503)
  [beaster]    > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:346)
  [beaster]    > at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:424)
  [beaster]    > at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:237)
  [beaster]    > at org.apache.solr.cloud.RecoveryStrategy.doReplicateOnlyRecovery(RecoveryStrategy.java:382)
  [beaster]    > at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:328)
  [beaster]    > at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:307)
  [beaster]    > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
  [beaster]    > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
  [beaster]    > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  [beaster]    > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
  [beaster]    > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  [beaster]    > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  [beaster]    > at java.lang.Thread.run(Thread.java:748)
  [beaster]    >
  [beaster]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
  [beaster]    > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  [beaster]    > at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321)
  [beaster]    > at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330)
  [beaster]    > at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:225)
  [beaster]    > at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:267)
  [beaster]    > at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:420)
  [beaster]    > at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:237)
  [beaster]    > at org.apache.solr.cloud.RecoveryStrategy.doReplicateOnlyRecovery(RecoveryStrategy.java:382)
  [beaster]    > at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:328)
  [beaster]    > at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:307)
  [beaster]    > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
  [beaster]    > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
  [beaster]    > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  [beaster]    > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
  [beaster]    > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  [beaster]    > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  [beaster]    > at java.lang.Thread.run(Thread.java:748)
  [beaster]    >
  [beaster]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
  [beaster]    > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  [beaster]    > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
  [beaster]    > at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:359)
  [beaster]    > at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:738)
  [beaster]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:965)
  [beaster]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:874)
  [beaster]    > at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1182)
  [beaster]    > at org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:694)
  [beaster]    > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
  [beaster]    > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  [beaster]    > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
  [beaster]    > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  [beaster]    > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  [beaster]    > at java.lang.Thread.run(Thread.java:748)
  [beaster]    >
  [beaster]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
  [beaster]    > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  [beaster]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1052)
  [beaster]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:874)
  [beaster]    > at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1182)
  [beaster]    > at org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:694)
  [beaster]    > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
  [beaster]    > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  [beaster]    > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
  [beaster]    > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  [beaster]    > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  [beaster]    > at java.lang.Thread.run(Thread.java:748)
  [beaster]    >
  [beaster]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
  [beaster]    > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  [beaster]    > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
  [beaster]    > at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:508)
  [beaster]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:959)
  [beaster]    > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:874)
  [beaster]    > at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1182)
  [beaster]    > at org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:694)
  [beaster]    > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
  [beaster]    > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  [beaster]    > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
  [beaster]    > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  [beaster]    > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  [beaster]    > at java.lang.Thread.run(Thread.java:748)
  [beaster]    >
  [beaster]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
  [beaster]    > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  [beaster]    > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
  [beaster]    > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:503)
  [beaster]    > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:346)
  [beaster]    > at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:424)
  [beaster]    > at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1184)
  [beaster]    > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
  [beaster]    > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
  [beaster]    > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
  [beaster]    > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
  [beaster]    > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  [beaster]    > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  [beaster]    > at java.lang.Thread.run(Thread.java:748)
  [beaster]    >
  [beaster]    >
  [beaster]    > at __randomizedtesting.SeedInfo.seed([65C44CE967C0F88A]:0)
  [beaster]    > at org.junit.Assert.fail(Assert.java:93)
  [beaster]    > at org.junit.Assert.assertTrue(Assert.java:43)
  [beaster]    > at org.junit.Assert.assertNull(Assert.java:551)
  [beaster]    > at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:317)
  [beaster]    > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  [beaster]    > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  [beaster]    > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  [beaster]    > at java.lang.reflect.Method.invoke(Method.java:498)
  [beaster]    > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
  [beaster]    > at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
  [beaster]    > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  [beaster]    > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
  [beaster]    > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
  [beaster]    > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  [beaster]    > at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
  [beaster]    > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  [beaster]    > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  [beaster]    > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  [beaster]    > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  [beaster]    > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  [beaster]    > at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  [beaster]    > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
  [beaster]    > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
  [beaster]    > at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
  [beaster]    > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  [beaster]    > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
  [beaster]    > at java.lang.Thread.run(Thread.java:748)
  [beaster]
  [beaster] Tests with failures [seed: 65C44CE967C0F88A]:
  [beaster]   - org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest (suite)
  [beaster]
  [beaster]
{noformat}

> Harden ChaosMonkeySafeLeaderWithPullReplicasTest
> ------------------------------------------------
>
>                 Key: SOLR-13031
>                 URL: https://issues.apache.org/jira/browse/SOLR-13031
>             Project: Solr
>          Issue Type: Sub-task
>      Security Level: Public(Default Security Level. Issues are Public)
>            Reporter: Mark Miller
>            Priority: Major
>
> This test can rarely leak objects tracked by the Test Object Tracker.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]