[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+181) - Build # 20294 - Unstable!

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+181) - Build # 20294 - Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/20294/
Java: 64bit/jdk-9-ea+181 -XX:-UseCompressedOops -XX:+UseParallelGC --illegal-access=deny

2 tests failed.
FAILED:  org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenRenew

Error Message:
expected:<200> but was:<403>

Stack Trace:
java.lang.AssertionError: expected:<200> but was:<403>
        at __randomizedtesting.SeedInfo.seed([8AD1266E4C07868A:BD4AD27074CB5B2E]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.renewDelegationToken(TestSolrCloudWithDelegationTokens.java:130)
        at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.verifyDelegationTokenRenew(TestSolrCloudWithDelegationTokens.java:316)
        at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenRenew(TestSolrCloudWithDelegationTokens.java:333)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.core.TestJmxIntegration.testJmxRegistration

Error Message:
org.apache.lucene.store.AlreadyClosedException: Already closed

Stack Trace:
javax.management.RuntimeMBeanException: org.apache.lucene.store.AlreadyClosedException: Already closed
        at __randomizedtesting.SeedInfo.seed([8AD1266E4C07868A:40042542146DEEF]:0)
        at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:829)
        at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:842)
        at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:645)
        at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
        at org.apache.solr.core.TestJmxIntegration.testJmxRegistration(TestJmxIntegration.java:120)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: org.apache.lucene.store.AlreadyClosedException: Already closed
        at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:337)
        at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:349)
        at org.apache.solr.core.SolrCore.getIndexDir(SolrCore.java:328)
        at org.apache.solr.handler.ReplicationHandler.lambda$initializeMetrics$5(ReplicationHandler.java:873)
        at com.codahale.metrics.JmxReporter$JmxGauge.getValue(JmxReporter.java:188)
        at jdk.internal.reflect.GeneratedMethodAccessor93.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
        at jdk.internal.reflect.GeneratedMethodAccessor85.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at java.base/sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:260)
        at java.management/com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
        at java.management/com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
        at java.management/com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
        at java.management/com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
        at java.management/com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
        at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:641)
        ... 40 more




Build Log:
[...truncated 11375 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithDelegationTokens
   [junit4]   2> 552887 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_8AD1266E4C07868A-001/init-core-data-001
   [junit4]   2> 552887 WARN  (SUITE-TestSolrCloudWithDelegationTokens-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=128 numCloses=128
   [junit4]   2> 552888 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 552889 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
   [junit4]   2> 552890 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_8AD1266E4C07868A-001/tempDir-001
   [junit4]   2> 552890 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 552895 INFO  (Thread-3228) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 552895 INFO  (Thread-3228) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 552933 ERROR (Thread-3228) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 552995 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:36827
   [junit4]   2> 553017 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15dcdf837bc0000, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 553023 INFO  (jetty-launcher-543-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 553026 INFO  (jetty-launcher-543-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 553046 INFO  (jetty-launcher-543-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6759a986{/solr,null,AVAILABLE}
   [junit4]   2> 553047 INFO  (jetty-launcher-543-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4cd91493{SSL,[ssl, http/1.1]}{127.0.0.1:41307}
   [junit4]   2> 553047 INFO  (jetty-launcher-543-thread-1) [    ] o.e.j.s.Server Started @554897ms
   [junit4]   2> 553047 INFO  (jetty-launcher-543-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41307}
   [junit4]   2> 553047 ERROR (jetty-launcher-543-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 553047 INFO  (jetty-launcher-543-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 553047 INFO  (jetty-launcher-543-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 553047 INFO  (jetty-launcher-543-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 553047 INFO  (jetty-launcher-543-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-10T21:06:22.926367Z
   [junit4]   2> 553108 INFO  (jetty-launcher-543-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6f43f5cf{/solr,null,AVAILABLE}
   [junit4]   2> 553109 INFO  (jetty-launcher-543-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4d18cd31{SSL,[ssl, http/1.1]}{127.0.0.1:34721}
   [junit4]   2> 553109 INFO  (jetty-launcher-543-thread-2) [    ] o.e.j.s.Server Started @554959ms
   [junit4]   2> 553109 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34721}
   [junit4]   2> 553109 ERROR (jetty-launcher-543-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 553109 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 553109 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 553109 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 553109 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-10T21:06:22.988859Z
   [junit4]   2> 553141 INFO  (jetty-launcher-543-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 553144 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 553177 INFO  (jetty-launcher-543-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36827/solr
   [junit4]   2> 553207 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36827/solr
   [junit4]   2> 553304 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 553304 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34721_solr
   [junit4]   2> 553304 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.c.Overseer Overseer (id=98461232842670086-127.0.0.1:34721_solr-n_0000000000) starting
   [junit4]   2> 553329 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34721_solr
   [junit4]   2> 553330 INFO  (jetty-launcher-543-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 553338 INFO  (zkCallback-555-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 553339 INFO  (jetty-launcher-543-thread-1) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 553340 INFO  (jetty-launcher-543-thread-1) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41307_solr
   [junit4]   2> 553344 INFO  (zkCallback-554-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 553345 INFO  (zkCallback-555-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 553355 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
   [junit4]   2> 553356 INFO  (jetty-launcher-543-thread-1) [    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
   [junit4]   2> 553359 WARN  (jetty-launcher-543-thread-2) [    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 553369 WARN  (jetty-launcher-543-thread-1) [    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 553371 INFO  (jetty-launcher-543-thread-1) [    ] o.a.s.s.KerberosPlugin Params: {token.valid=30, delegation-token.token-kind=solr-dt, cookie.domain=127.0.0.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, token.validity=36000, signer.secret.provider.zookeeper.path=/token, type=kerberos, zk-dt-secret-manager.enable=true, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper}
   [junit4]   2> 553377 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.s.KerberosPlugin Params: {token.valid=30, delegation-token.token-kind=solr-dt, cookie.domain=127.0.0.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, token.validity=36000, signer.secret.provider.zookeeper.path=/token, type=kerberos, zk-dt-secret-manager.enable=true, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper}
   [junit4]   2> 553444 INFO  (jetty-launcher-543-thread-2) [    ] o.a.c.f.i.CuratorFrameworkImpl Starting
   [junit4]   2> 553449 INFO  (jetty-launcher-543-thread-1) [    ] o.a.c.f.i.CuratorFrameworkImpl Starting
   [junit4]   2> 553501 INFO  (jetty-launcher-543-thread-1-EventThread) [    ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
   [junit4]   2> 553503 INFO  (jetty-launcher-543-thread-2-EventThread) [    ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
   [junit4]   2> 553591 WARN  (jetty-launcher-543-thread-1) [    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 553630 WARN  (jetty-launcher-543-thread-2) [    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 553657 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34721.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34e8f8d5
   [junit4]   2> 553661 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34721.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34e8f8d5
   [junit4]   2> 553661 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34721.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34e8f8d5
   [junit4]   2> 553661 INFO  (jetty-launcher-543-thread-1) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41307.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34e8f8d5
   [junit4]   2> 553662 INFO  (jetty-launcher-543-thread-2) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_8AD1266E4C07868A-001/tempDir-001/node2/.
   [junit4]   2> 553664 INFO  (jetty-launcher-543-thread-1) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41307.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34e8f8d5
   [junit4]   2> 553664 INFO  (jetty-launcher-543-thread-1) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41307.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@34e8f8d5
   [junit4]   2> 553665 INFO  (jetty-launcher-543-thread-1) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_8AD1266E4C07868A-001/tempDir-001/node1/.
   [junit4]   2> 553799 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenRenewFail
   [junit4]   2> 553866 WARN  (qtp85702771-6295) [    ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: foo tries to renew a token with renewer bar
   [junit4]   2> 553871 WARN  (qtp85702771-6295) [    ] o.a.s.s.KerberosPlugin Could not find org.apache.solr.security.authentication.requestcontinues
   [junit4]   2> 553874 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[8AD1266E4C07868A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=do, 01-jan.-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: do, 01-jan.-1970 00:00:00 GMT
   [junit4]   2> 553900 WARN  (qtp678257949-6278) [    ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: foo tries to renew a token with renewer bar
   [junit4]   2> 553901 WARN  (qtp678257949-6278) [    ] o.a.s.s.KerberosPlugin Could not find org.apache.solr.security.authentication.requestcontinues
   [junit4]   2> 553901 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[8AD1266E4C07868A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=do, 01-jan.-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: do, 01-jan.-1970 00:00:00 GMT
   [junit4]   2> 553910 WARN  (qtp85702771-6294) [    ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: notRenewUser tries to renew a token with renewer renewUser
   [junit4]   2> 553910 WARN  (qtp85702771-6294) [    ] o.a.s.s.KerberosPlugin Could not find org.apache.solr.security.authentication.requestcontinues
   [junit4]   2> 553913 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[8AD1266E4C07868A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=do, 01-jan.-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: do, 01-jan.-1970 00:00:00 GMT
   [junit4]   2> 553914 WARN  (qtp678257949-6284) [    ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: notRenewUser tries to renew a token with renewer renewUser
   [junit4]   2> 553914 WARN  (qtp678257949-6284) [    ] o.a.s.s.KerberosPlugin Could not find org.apache.solr.security.authentication.requestcontinues
   [junit4]   2> 553915 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[8AD1266E4C07868A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=do, 01-jan.-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: do, 01-jan.-1970 00:00:00 GMT
   [junit4]   2> 553915 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenewFail-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenRenewFail
   [junit4]   2> 553927 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenCancel
   [junit4]   2> 553980 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[8AD1266E4C07868A]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 553981 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[8AD1266E4C07868A]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36827/solr ready
   [junit4]   2> 553987 ERROR (qtp85702771-6294) [    ] o.a.h.s.t.d.ZKDelegationTokenSecretManager No node in path [/ZKDTSMTokensRoot/DT_3]
   [junit4]   2> 553994 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[8AD1266E4C07868A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=do, 01-jan.-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: do, 01-jan.-1970 00:00:00 GMT
   [junit4]   2> 553999 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[8AD1266E4C07868A]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 553999 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[8AD1266E4C07868A]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36827/solr ready
   [junit4]   2> 554003 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[8AD1266E4C07868A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=do, 01-jan.-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: do, 01-jan.-1970 00:00:00 GMT
   [junit4]   2> 554033 ERROR (qtp678257949-6285) [    ] o.a.h.s.t.d.ZKDelegationTokenSecretManager No node in path [/ZKDTSMTokensRoot/DT_4]
   [junit4]   2> 554058 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[8AD1266E4C07868A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=do, 01-jan.-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: do, 01-jan.-1970 00:00:00 GMT
   [junit4]   2> 554063 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[8AD1266E4C07868A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=do, 01-jan.-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: do, 01-jan.-1970 00:00:00 GMT
   [junit4]   2> 554064 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenCancel-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenCancel
   [junit4]   2> 554078 INFO  (TEST-TestSolrCloudWithDelegationTokens.testZNodePaths-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testZNodePaths
   [junit4]   2> 554125 INFO  (TEST-TestSolrCloudWithDelegationTokens.testZNodePaths-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testZNodePaths
   [junit4]   2> 554134 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenSolrClient-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenSolrClient
   [junit4]   2> 554140 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenSolrClient-seed#[8AD1266E4C07868A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=do, 01-jan.-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: do, 01-jan.-1970 00:00:00 GMT
   [junit4]   2> 554152 INFO  (qtp85702771-6294) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAV3N-DyNigFd8gTAjQYEFOPhPcOxFPZ_51tsJVb9L2KJTJc7B3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 554152 INFO  (qtp85702771-6294) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAV3N-DyNigFd8gTAjQYEFOPhPcOxFPZ_51tsJVb9L2KJTJc7B3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 554153 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenSolrClient-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenSolrClient
   [junit4] IGNOR/A 0.00s J0 | TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail
   [junit4]    > Assumption #1: 'awaitsfix' test group is disabled (@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/HADOOP-14044"))
   [junit4]   2> 554186 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationOtherOp-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationOtherOp
   [junit4]   2> 554193 INFO  (qtp85702771-6291) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params op=someSolrOperation&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 554194 INFO  (qtp85702771-6291) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={op=someSolrOperation&action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 554194 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationOtherOp-seed#[8AD1266E4C07868A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bar&p=bar&t=test&e=1502435184071&s=dkbodmnPN9Qn4Ne5is7y0L8R5eg="; Path=/; Domain=127.0.0.1; Expires=vr, 11-aug.-2017 07:06:24 GMT; Secure; HttpOnly". Invalid 'expires' attribute: vr, 11-aug.-2017 07:06:24 GMT
   [junit4]   2> 554194 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationOtherOp-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationOtherOp
   [junit4]   2> 554203 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenVerify
   [junit4]   2> 554250 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[8AD1266E4C07868A]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 554250 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[8AD1266E4C07868A]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36827/solr ready
   [junit4]   2> 554254 INFO  (qtp85702771-6290) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAV3N-DzTigFd8gTA0wcEFP4AIqD-n_hxTbnV_SFASkZae1bEB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 554254 INFO  (qtp85702771-6290) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAV3N-DzTigFd8gTA0wcEFP4AIqD-n_hxTbnV_SFASkZae1bEB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 554270 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[8AD1266E4C07868A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=do, 01-jan.-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: do, 01-jan.-1970 00:00:00 GMT
   [junit4]   2> 554291 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[8AD1266E4C07868A]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 554291 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[8AD1266E4C07868A]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36827/solr ready
   [junit4]   2> 554297 INFO  (qtp85702771-6291) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&delegation=GgADYmFyA2JhcgCKAV3N-DzTigFd8gTA0wcEFP4AIqD-n_hxTbnV_SFASkZae1bEB3NvbHItZHQA&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 554297 INFO  (qtp85702771-6291) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&delegation=GgADYmFyA2JhcgCKAV3N-DzTigFd8gTA0wcEFP4AIqD-n_hxTbnV_SFASkZae1bEB3NvbHItZHQA&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 554301 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[8AD1266E4C07868A]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 554302 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[8AD1266E4C07868A]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36827/solr ready
   [junit4]   2> 554306 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[8AD1266E4C07868A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=do, 01-jan.-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: do, 01-jan.-1970 00:00:00 GMT
   [junit4]   2> 554306 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenVerify-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenVerify
   [junit4]   2> 554319 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDelegationTokenRenew
   [junit4]   2> 554326 WARN  (qtp85702771-6290) [    ] o.a.h.s.a.s.AuthenticationFilter Authentication exception: org.apache.hadoop.security.AccessControlException: bar is trying to renew a token with wrong password
   [junit4]   2> 554326 WARN  (qtp85702771-6290) [    ] o.a.s.s.KerberosPlugin Could not find org.apache.solr.security.authentication.requestcontinues
   [junit4]   2> 554326 WARN  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[8AD1266E4C07868A]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth=; Path=/; Domain=127.0.0.1; Expires=do, 01-jan.-1970 00:00:00 GMT; Secure; HttpOnly". Invalid 'expires' attribute: do, 01-jan.-1970 00:00:00 GMT
   [junit4]   2> 554326 INFO  (TEST-TestSolrCloudWithDelegationTokens.testDelegationTokenRenew-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testDelegationTokenRenew
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSolrCloudWithDelegationTokens -Dtests.method=testDelegationTokenRenew -Dtests.seed=8AD1266E4C07868A -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=nl-CW -Dtests.timezone=Europe/Tallinn -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.02s J0 | TestSolrCloudWithDelegationTokens.testDelegationTokenRenew <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: expected:<200> but was:<403>
   [junit4]    > at __randomizedtesting.SeedInfo.seed([8AD1266E4C07868A:BD4AD27074CB5B2E]:0)
   [junit4]    > at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.renewDelegationToken(TestSolrCloudWithDelegationTokens.java:130)
   [junit4]    > at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.verifyDelegationTokenRenew(TestSolrCloudWithDelegationTokens.java:316)
   [junit4]    > at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenRenew(TestSolrCloudWithDelegationTokens.java:333)
   [junit4]    > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > at java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]    > at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 554329 INFO  (jetty-closer-544-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@4d18cd31{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 554329 INFO  (jetty-closer-544-thread-2) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@4cd91493{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 554329 INFO  (jetty-closer-544-thread-2) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1860169729
   [junit4]   2> 554330 INFO  (jetty-closer-544-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 554331 INFO  (jetty-closer-544-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 554331 INFO  (jetty-closer-544-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=46830898
   [junit4]   2> 554331 INFO  (jetty-closer-544-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 554333 INFO  (jetty-closer-544-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 554334 INFO  (jetty-closer-544-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 554334 INFO  (jetty-closer-544-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 554336 INFO  (jetty-closer-544-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 554336 INFO  (jetty-closer-544-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 554336 INFO  (jetty-closer-544-thread-2) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 554336 INFO  (jetty-closer-544-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 554336 INFO  (jetty-closer-544-thread-1) [    ] o.a.s.c.Overseer Overseer (id=98461232842670086-127.0.0.1:34721_solr-n_0000000000) closing
   [junit4]   2> 554336 INFO  (OverseerStateUpdate-98461232842670086-127.0.0.1:34721_solr-n_0000000000) [    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:34721_solr
   [junit4]   2> 554337 INFO  (zkCallback-555-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 554341 ERROR (Thread[Thread-3245,5,TGRP-TestSolrCloudWithDelegationTokens]) [    ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 554349 INFO  (jetty-closer-544-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6759a986{/solr,null,UNAVAILABLE}
   [junit4]   2> 555842 WARN  (zkCallback-555-thread-1) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 555845 ERROR (Thread[Thread-3247,5,TGRP-TestSolrCloudWithDelegationTokens]) [    ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 555846 INFO  (jetty-closer-544-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6f43f5cf{/solr,null,UNAVAILABLE}
   [junit4]   2> 555849 ERROR (SUITE-TestSolrCloudWithDelegationTokens-seed#[8AD1266E4C07868A]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 555849 INFO  (SUITE-TestSolrCloudWithDelegationTokens-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36827 36827
   [junit4]   2> 557085 INFO  (Thread-3228) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36827 36827
   [junit4]   2> 557086 WARN  (Thread-3228) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 7 /solr/aliases.json
   [junit4]   2> 2 /solr/security.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 7 /solr/clusterprops.json
   [junit4]   2> 7 /solr/clusterstate.json
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_7
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_6
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_8
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMKeyIdRoot
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_1
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_1
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_3
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_2
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_2
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_5
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_4
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot/DT_4
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMSeqNumRoot
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 7 /solr/live_nodes
   [junit4]   2> 7 /solr/collections
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot
   [junit4]   2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_8AD1266E4C07868A-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=70, maxMBSortInHeap=5.593626499815866, sim=RandomSimilarity(queryNorm=true): {}, locale=nl-CW, timezone=Europe/Tallinn
   [junit4]   2> NOTE: Linux 4.10.0-27-generic amd64/Oracle Corporation 9 (64-bit)/cpus=8,threads=1,free=33234864,total=506986496
   [junit4]   2> NOTE: All tests run in this JVM: [TestFaceting, UUIDFieldTest, TestConfigOverlay, IndexBasedSpellCheckerTest, ZkStateReaderTest, TestSystemCollAutoCreate, SolrIndexConfigTest, PeerSyncTest, LeaderElectionIntegrationTest, TestClassicSimilarityFactory, TestNumericRangeQuery32, TestLeaderInitiatedRecoveryThread, BasicZkTest, TestCrossCoreJoin, TestSolrQueryParser, TestCloudSearcherWarming, SuggestComponentTest, TestManagedSynonymGraphFilterFactory, ConnectionReuseTest, TestUninvertingReader, HttpSolrCallGetCoreTest, HdfsWriteToMultipleCollectionsTest, DefaultValueUpdateProcessorTest, TestRuleBasedAuthorizationPlugin, HdfsBasicDistributedZkTest, TestDistribIDF, TestCloudPivotFacet, SparseHLLTest, TestCopyFieldCollectionResource, TestHdfsUpdateLog, HdfsDirectoryFactoryTest, UpdateLogTest, TestPerFieldSimilarityWithDefaultOverride, TestHighlightDedupGrouping, TestUseDocValuesAsStored2, TestSubQueryTransformerCrossCore, HdfsCollectionsAPIDistributedZkTest, TestIndexingPerformance, TestRecoveryHdfs, TestRequestForwarding, ExternalFileFieldSortTest, TestFileDictionaryLookup, RankQueryTest, TestTlogReplica, TestHttpShardHandlerFactory, TestMinMaxOnMultiValuedField, TestSolrCloudWithDelegationTokens]
   [junit4] Completed [194/729 (1!)] on J0 in 4.28s, 8 tests, 1 failure, 1 skipped <<< FAILURES!

[...truncated 799 lines...]
   [junit4] Suite: org.apache.solr.core.TestJmxIntegration
   [junit4]   2> 1241848 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.core.TestJmxIntegration_8AD1266E4C07868A-001/init-core-data-001
   [junit4]   2> 1241848 WARN  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=3 numCloses=3
   [junit4]   2> 1241848 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1241849 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
   [junit4]   2> 1241849 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 1241849 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1241862 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1241873 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 1241937 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1241954 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7bb142d
   [junit4]   2> 1241956 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7bb142d
   [junit4]   2> 1241956 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7bb142d
   [junit4]   2> 1241957 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1241971 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1241981 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1242068 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1242074 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 1242074 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7bb142d
   [junit4]   2> 1242074 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1242074 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.core.TestJmxIntegration_8AD1266E4C07868A-001/init-core-data-001/]
   [junit4]   2> 1242075 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=14.5400390625, floorSegmentMB=1.65234375, forceMergeDeletesPctAllowed=23.2345209213156, segmentsPerTier=38.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1242106 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1242106 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1242106 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1242106 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1242107 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=19, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1242107 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@744ec04b[collection1] main]
   [junit4]   2> 1242107 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1242118 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1242189 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1242189 INFO  (searcherExecutor-6376-thread-1) [    ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1242189 INFO  (coreLoadExecutor-6375-thread-1) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575380447971508224
   [junit4]   2> 1242189 INFO  (searcherExecutor-6376-thread-1) [    ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 1242190 INFO  (searcherExecutor-6376-thread-1) [    ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1242190 INFO  (searcherExecutor-6376-thread-1) [    ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1242190 INFO  (searcherExecutor-6376-thread-1) [    ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1242190 INFO  (searcherExecutor-6376-thread-1) [    ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 1242190 INFO  (searcherExecutor-6376-thread-1) [    ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 1242190 INFO  (searcherExecutor-6376-thread-1) [    ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 1242190 INFO  (searcherExecutor-6376-thread-1) [    ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 1242191 INFO  (searcherExecutor-6376-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@744ec04b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1242191 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 1242192 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testJmxUpdate
   [junit4]   2> 1242208 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[8AD1266E4C07868A]) [    ] o.a.s.c.TestJmxIntegration Mbeans in server: [solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Survivor-Space,name=max, java.lang:type=Threading, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mock,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/get,name=errors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=requestTimes, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.updateExecutor.running, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mock,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/stream,name=requestTimes, solr:dom1=node,category=QUERY,scope=httpShardHandler,name=threadPool.httpShardExecutor.duration, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=handlerStart, solr:dom1=node,category=ADMIN,scope=/admin/health,name=errors, solr:dom1=jvm,name0=buffers,name1=mapped,name=TotalCapacity, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=totalTime, solr:dom1=jvm,name0=memory,name1=pools,name2=CodeHeap-'non-profiled-nmethods',name=usage, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/luke,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/logging,name=requests, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=rollbacks, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/schema,name=errors, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/stream,name=requestTimes, solr:dom1=node,category=CONTAINER,scope=threadPool,name=coreContainerWorkExecutor.running, solr:dom1=core,dom2=collection1,category=QUERY,scope=/terms,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=/update/json/docs,name=timeouts, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=requestTimes, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragmenter,name=gap.requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/export,name=totalTime, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=/update,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=errors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/plugins,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=updateHandler,name=cumulativeDeletesByQuery, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=SEARCHER,scope=new,name=warmup, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=TLOG,scope=replay,name=ops, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/segments,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/graph,name=errors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/threads,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=errors, solr:dom1=jvm,name0=threads,name1=waiting,name=count, solr:dom1=jvm,name0=classes,name=unloaded, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/properties,name=timeouts, solr:dom1=node,category=CONTAINER,scope=cores,name=loaded, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=handlerStart, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=HIGHLIGHTER,scope=SolrFragListBuilder,name=default.requests, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=totalTime, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=new,name=errors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=SEARCHER,scope=new,name=maxReached, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/config,name=clientErrors, solr:dom1=node,category=QUERY,scope=httpShardHandler,name=threadPool.httpShardExecutor.completed, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/config,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/debug/dump,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak_Direct,name=serverErrors, java.lang:type=Runtime, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/config,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/defaults,name=timeouts, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=errors, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.recoveryExecutor.duration, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak_Direct,name=timeouts, solr:dom1=core,dom2=collection1,category=CORE,scope=fs,name=totalSpace, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/sql,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=TLOG,scope=replay,name=remaining.logs, solr:dom1=node,category=CONTAINER,scope=threadPool,name=coreContainerWorkExecutor.duration, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=update,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak_Direct,name=clientErrors, solr:dom1=jvm,name0=threads,name1=termınated,name=count, solr:dom1=node,category=CONTAINER,scope=cores,name=unloaded, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=serverErrors, solr:dom1=node,category=ADMIN,scope=/admin/metrics,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH,name=serverErrors, solr:dom1=jvm,name0=os,name=totalSwapSpaceSize, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=cumulativeErrors, java.lang:name=PS Survivor Space,type=MemoryPool, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH,name=errors, solr:dom1=jvm,name0=os,name=availableProcessors, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.recoveryExecutor.running, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/segments,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak,name=errors, solr:dom1=node,category=CONTAINER,scope=fs,name=path, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/terms,name=requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/luke,name=errors, solr:dom1=jvm,name0=threads,name=count, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=caching, solr:dom1=jvm,name0=memory,name1=total,name=used, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFormatter,name=html.requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/threads,name=requestTimes, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/luke,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak,name=handlerStart, solr:dom1=core,dom2=collection1,category=TLOG,scope=replay,name=ops, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH1,name=handlerStart, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=serverErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=timeouts, solr:dom1=jvm,name0=threads,name1=waıtıng,name=count, solr:dom1=node,category=ADMIN,scope=/admin/configs,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mock,name=initCount, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/file,name=clientErrors, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/plugins,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/select,name=requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/select,name=timeouts, solr:dom1=jvm,name0=memory,name1=pools,name2=CodeHeap-'profiled-nmethods',name=committed, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=serverErrors, solr:dom1=node,category=QUERY,scope=/admin/autoscaling,name=handlerStart, solr:dom1=node,category=ADMIN,scope=/admin/zookeeper,name=serverErrors, solr:dom1=node,category=CONTAINER,scope=fs,name=coreRoot.path, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=errors, solr:dom1=node,category=ADMIN,scope=/admin/configs,name=requests, solr:dom1=jvm,name0=memory,name1=pools,name2=CodeHeap-'non-nmethods',name=max, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/get,name=errors, solr:dom1=node,category=CONTAINER,scope=threadPool,name=coreLoadExecutor.duration, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/get,name=clientErrors, solr:dom1=jvm,name0=memory,name1=pools,name2=CodeHeap-'non-nmethods',name=committed, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/threads,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/stream,name=requests, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/file,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/plugins,name=serverErrors, java.lang:name=CodeHeap 'non-profiled nmethods',type=MemoryPool, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/graph,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/graph,name=requests, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=requestTimes, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/schema,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=/update/json/docs,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=timeouts, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH1,name=clientErrors, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=timeouts, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=adds, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/properties,name=errors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/terms,name=requestTimes, solr:dom1=jvm,name0=os,name=version, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=serverErrors, solr:dom1=jvm,name0=memory,name1=pools,name2=Metaspace,name=usage, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/select,name=requests, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mlt,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak_Direct,name=handlerStart, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=errors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/properties,name=totalTime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/export,name=timeouts, solr:dom1=node,category=QUERY,scope=/admin/autoscaling,name=errors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/plugins,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/terms,name=requests, solr:dom1=jvm,name0=os,name=maxFileDescriptorCount, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragmentsBuilder,name=simple.requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/get,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/file,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/logging,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/sql,name=errors, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=confFilesToReplicate, solr:dom1=node,category=CONTAINER,scope=fs,name=coreRoot.usableSpace, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragListBuilder,name=default.requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=/update/json/docs,name=requests, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=splits, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/config,name=errors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/terms,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=/update/json/docs,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/properties,name=requestTimes, solr:dom1=jvm,name0=memory,name1=pools,name2=CodeHeap-'profiled-nmethods',name=max, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=fetcher, solr:dom1=core,dom2=collection1,category=QUERY,scope=/graph,name=timeouts, solr:dom1=jvm,name0=threads,name1=blocked,name=count, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=totalTime, solr:dom1=jvm,name0=gc,name1=PS-MarkSweep,name=time, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/terms,name=errors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/segments,name=serverErrors, solr:dom1=node,category=ADMIN,scope=/admin/info,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/select,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/segments,name=errors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/get,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/export,name=serverErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/logging,name=totalTime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replic
   [junit4]   2> a_n41,category=ADMIN,scope=/schema,name=requests, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/luke,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=totalTime, solr:dom1=jvm,name0=threads,name1=tımed_waıtıng,name=count, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/dump,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=REPLICATION,scope=/replication,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mock,name=clientErrors, solr:dom1=jvm,name0=threads,name1=terminated,name=count, java.lang:name=Metaspace Manager,type=MemoryManager, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/stream,name=errors, solr:dom1=node,category=QUERY,scope=/admin/metrics/collector,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH1,name=errors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/graph,name=requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/mbeans,name=timeouts, java.lang:name=CodeHeap 'profiled nmethods',type=MemoryPool, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/dump,name=totalTime, solr:dom1=jvm,name0=memory,name1=pools,name2=CodeHeap-'non-nmethods',name=init, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=update,name=errors, solr:dom1=node,category=CONTAINER,scope=threadPool,name=coreContainerWorkExecutor.completed, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/luke,name=serverErrors, solr:dom1=node,category=QUERY,scope=/admin/autoscaling,name=requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=/update/csv,name=requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/schema,name=timeouts, solr:dom1=jvm,name0=memory,name1=pools,name2=Metaspace,name=max, solr:dom1=node,category=ADMIN,scope=/admin/info,name=requestTimes, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=isSlave, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=requests, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=serverErrors, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/mbeans,name=timeouts, solr:dom1=jvm,name0=os,name=freeSwapSpaceSize, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mltrh,name=clientErrors, solr:dom1=node,category=QUERY,scope=httpShardHandler,name=availableConnections, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=indexPath, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/debug/dump,name=requestTimes, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Eden-Space,name=committed, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=requests, solr:dom1=node,category=ADMIN,scope=/admin/configs,name=handlerStart, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=requestTimes, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Old-Gen,name=used-after-gc, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=timeouts, java.lang:name=PS Scavenge,type=GarbageCollector, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/file,name=serverErrors, solr:dom1=node,category=ADMIN,scope=/admin/info,name=clientErrors, solr:dom1=jvm,name0=memory,name1=heap,name=usage, java.nio:name=mapped,type=BufferPool, com.sun.management:type=DiagnosticCommand, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Eden-Space,name=used-after-gc, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=replicationEnabled, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=new,name=maxReached, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=REPLICATION,scope=/replication,name=clientErrors, solr:dom1=node,category=ADMIN,scope=/admin/info,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=clientErrors, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=threadPool.parallelCoreAdminExecutor.running, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.updateExecutor.duration, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=serverErrors, solr:dom1=node,category=ADMIN,scope=/admin/health,name=handlerStart, solr:dom1=node,category=CONTAINER,scope=fs,name=usableSpace, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH1,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=updateHandler,name=optimizes, solr:dom1=node,category=ADMIN,scope=/admin/metrics,name=errors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=merges, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/debug/dump,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/tvrh,name=serverErrors, solr:dom1=node,category=CONTAINER,scope=version,name=specification, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=errors, solr:dom1=jvm,name0=threads,name1=timed_waiting,name=count, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=/update,name=requests, solr:dom1=core,dom2=collection1,category=CORE,name=instanceDir, solr:dom1=core,dom2=collection1,category=QUERY,scope=/tvrh,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/file,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=handlerStart, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json,name=clientErrors, solr:dom1=jvm,name0=memory,name1=pools,name2=Metaspace,name=used, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/ping,name=requestTimes, solr:dom1=jvm,name0=memory,name1=pools,name2=CodeHeap-'profiled-nmethods',name=used, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/select,name=errors, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=readerDir, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/luke,name=timeouts, solr:dom1=node,category=ADMIN,scope=/admin/zookeeper,name=requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/schema,name=requestTimes, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/mbeans,name=requests, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/select,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=requests, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=clientErrors, solr:dom1=node,category=CONTAINER,scope=threadPool,name=coreContainerWorkExecutor.submitted, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/threads,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=/update/csv,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/schema,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/system,name=timeouts, solr:dom1=core,dom2=collection1,category=CACHE,scope=core,name=fieldCache, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/schema,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH_Direct,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/terms,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/select,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=/update/json/docs,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=REPLICATION,scope=/replication,name=generation, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/file,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/logging,name=requestTimes, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Survivor-Space,name=committed, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=/update/json,name=timeouts, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragmenter,name=default.requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/tvrh,name=requests, solr:dom1=node,category=ADMIN,scope=/admin/info,name=requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/graph,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mltrh,name=timeouts, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragmentsBuilder,name=default.requests, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=docsPending, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Survivor-Space,name=used-after-gc, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/select,name=clientErrors, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=serverErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/export,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH1,name=totalTime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/logging,name=requestTimes, solr:dom1=jvm,name0=memory,name1=total,name=max, solr:dom1=core,dom2=collection1,category=CORE,name=aliases, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/dump,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/debug/dump,name=errors, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=requests, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=registeredAt, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=update,name=requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=/update/csv,name=requestTimes, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/file,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mltrh,name=handlerStart, solr:dom1=node,category=ADMIN,scope=/admin/info,name=serverErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=handlerStart, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Survivor-Space,name=used, solr:dom1=core,dom2=collection1,category=CORE,name=startTime, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/plugins,name=requests, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json/docs,name=requestTimes, java.lang:name=PS Old Gen,type=MemoryPool, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Survivor-Space,name=init, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/defaults,name=requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/segments,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mltrh,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=REPLICATION,scope=/replication,name=handlerStart, com.sun.management:type=HotSpotDiagnostic, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=HIGHLIGHTER,scope=SolrBoundaryScanner,name=default.requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH,name=requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/graph,name=requestTimes, solr:dom1=jvm,name0=memory,name1=non-heap,name=committed, solr:dom1=jvm,name0=buffers,name1=mapped,name=MemoryUsed, solr:dom1=jvm,name0=buffers,name1=direct,name=Count, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=updateHandler,name=commits, solr:dom1=jvm,name0=os,name=processCpuTime, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/threads,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/config,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/stream,name=errors, solr:dom1=jvm,name0=memory,name1=heap,name=committed, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/file,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/defaults,name=requestTimes, solr:dom1=jvm,name0=threads,name1=deadlock,name=count, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckCompRH1,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=errors, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=warmupTime, solr:dom1=jvm,name0=threads,name1=runnable,name=count, solr:dom1=node,category=ADMIN,scope=/admin/metrics,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/logging,name=errors, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=indexVersion, solr:dom1=jvm,name0=memory,name1=heap,name=init, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=numDocs, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak,name=requestTimes, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFormatter,name=default.requests, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragmenter,name=regex.requests, solr:dom1=node,category=ADMIN,scope=/admin/configs,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/select,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=update,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/threads,name=requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=HIGHLIGHTER,scope=SolrFragmentsBuilder,name=default.requests, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/config,name=requests, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=REPLICATION,scope=/replication,name=confFilesToReplicate, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/properties,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=updateHandler,name=expungeDeletes, java.lang:name=PS MarkSweep,type=GarbageCollector, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=/update,name=serverErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=ADMIN,scope=/admin/file,name=timeouts, solr:dom1=node,category=CONTAINER,scope=fs,name=coreRoot.totalSpace, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/system,name=requests, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=totalTime, solr:dom1=core,dom2=
   [junit4]   2> collection1,category=QUERY,scope=/debug/dump,name=requests, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json/docs,name=handlerStart, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/luke,name=handlerStart, solr:dom1=node,category=CONTAINER,scope=cores,name=lazy, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=handlerStart, solr:dom1=node,category=QUERY,scope=/admin/metrics/collector,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/defaults,name=handlerStart, java.lang:name=CodeHeap 'non-nmethods',type=MemoryPool, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=/update,name=requestTimes, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=optimizes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/tvrh,name=totalTime, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Eden-Space,name=init, solr:dom1=jvm,name0=os,name=openFileDescriptorCount, solr:dom1=node,category=QUERY,scope=httpShardHandler,name=pendingConnections, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/debug/dump,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/dump,name=errors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=errors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=update,name=totalTime, solr:dom1=node,category=CACHE,name=fieldCache, solr:dom1=core,dom2=collection1,category=QUERY,scope=/spellCheckWithWordbreak_Direct,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=QUERY,scope=/terms,name=timeouts, solr:dom1=jvm,name0=memory,name1=pools,name2=CodeHeap-'non-nmethods',name=usage, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mock,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/stream,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard2,dom4=replica_n41,category=UPDATE,scope=updateHandler,name=cumulativeAdds, solr:d

[...truncated too long message...]

browse/SOLR-2715"))
   [junit4]   2> 1242239 INFO  (TEST-TestJmxIntegration.testJmxRegistration-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testJmxRegistration
   [junit4]   2> 1242349 INFO  (TEST-TestJmxIntegration.testJmxRegistration-seed#[8AD1266E4C07868A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testJmxRegistration
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestJmxIntegration -Dtests.method=testJmxRegistration -Dtests.seed=8AD1266E4C07868A -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=fa-AF -Dtests.timezone=Asia/Hong_Kong -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.12s J2 | TestJmxIntegration.testJmxRegistration <<<
   [junit4]    > Throwable #1: javax.management.RuntimeMBeanException: org.apache.lucene.store.AlreadyClosedException: Already closed
   [junit4]    > at __randomizedtesting.SeedInfo.seed([8AD1266E4C07868A:40042542146DEEF]:0)
   [junit4]    > at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:829)
   [junit4]    > at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:842)
   [junit4]    > at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:645)
   [junit4]    > at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
   [junit4]    > at org.apache.solr.core.TestJmxIntegration.testJmxRegistration(TestJmxIntegration.java:120)
   [junit4]    > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > at java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]    > at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]    > Caused by: org.apache.lucene.store.AlreadyClosedException: Already closed
   [junit4]    > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:337)
   [junit4]    > at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:349)
   [junit4]    > at org.apache.solr.core.SolrCore.getIndexDir(SolrCore.java:328)
   [junit4]    > at org.apache.solr.handler.ReplicationHandler.lambda$initializeMetrics$5(ReplicationHandler.java:873)
   [junit4]    > at com.codahale.metrics.JmxReporter$JmxGauge.getValue(JmxReporter.java:188)
   [junit4]    > at jdk.internal.reflect.GeneratedMethodAccessor93.invoke(Unknown Source)
   [junit4]    > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > at java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]    > at jdk.internal.reflect.GeneratedMethodAccessor85.invoke(Unknown Source)
   [junit4]    > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > at java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]    > at java.base/sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:260)
   [junit4]    > at java.management/com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
   [junit4]    > at java.management/com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
   [junit4]    > at java.management/com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
   [junit4]    > at java.management/com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
   [junit4]    > at java.management/com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
   [junit4]    > at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:641)
   [junit4]    > ... 40 more
   [junit4]   2> 1242350 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 1242350 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1792374999
   [junit4]   2> 1242350 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1242352 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1242353 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1242353 INFO  (SUITE-TestJmxIntegration-seed#[8AD1266E4C07868A]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1242353 INFO  (coreCloseExecutor-6381-thread-1) [    ] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@7de1e8d6
   [junit4]   2> 1242357 INFO  (coreCloseExecutor-6381-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=2111957206
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.core.TestJmxIntegration_8AD1266E4C07868A-001
   [junit4]   2> Aug 10, 2017 9:17:51 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=6894, maxDocsPerChunk=5, blockSize=8), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=6894, blockSize=8)), sim=RandomSimilarity(queryNorm=false): {}, locale=fa-AF, timezone=Asia/Hong_Kong
   [junit4]   2> NOTE: Linux 4.10.0-27-generic amd64/Oracle Corporation 9 (64-bit)/cpus=8,threads=1,free=370572712,total=529006592
   [junit4]   2> NOTE: All tests run in this JVM: [TestBulkSchemaAPI, URLClassifyProcessorTest, V2StandaloneTest, DocValuesNotIndexedTest, NumericFieldsTest, TestReload, TestSort, TestObjectReleaseTracker, TestGroupingSearch, TestStressLiveNodes, ReturnFieldsTest, TestSolrDeletionPolicy2, MergeStrategyTest, TestDelegationWithHadoopAuth, TestPseudoReturnFields, PropertiesRequestHandlerTest, TestSSLRandomization, HdfsBasicDistributedZk2Test, TermsComponentTest, SuggesterFSTTest, TestSolrConfigHandlerConcurrent, TestTrie, TestDynamicFieldResource, CoreMergeIndexesAdminHandlerTest, TestCloudSchemaless, TestTolerantSearch, DistribJoinFromCollectionTest, CopyFieldTest, TestFoldingMultitermQuery, TestUtils, TestBlendedInfixSuggestions, TestBackupRepositoryFactory, TestCodecSupport, TestDeleteCollectionOnDownNodes, TestReRankQParserPlugin, TestOverriddenPrefixQueryForCustomFieldType, ConjunctionSolrSpellCheckerTest, TestConfigsApi, TestDFRSimilarityFactory, DirectUpdateHandlerOptimizeTest, TestQueryUtils, CircularListTest, SimpleMLTQParserTest, TestCloudNestedDocsSort, TestScoreJoinQPScore, TestReversedWildcardFilterFactory, SolrInfoBeanTest, DistributedExpandComponentTest, SolrGraphiteReporterTest, TemplateUpdateProcessorTest, SuggesterTest, TestRawTransformer, AnalysisErrorHandlingTest, TestComplexPhraseQParserPlugin, DeleteNodeTest, RecoveryAfterSoftCommitTest, TestSizeLimitedDistributedMap, TestTolerantUpdateProcessorRandomCloud, TestSolr4Spatial2, TestInPlaceUpdatesStandalone, ClusterStateTest, TestRecovery, TestNumericTerms64, SolrTestCaseJ4Test, SharedFSAutoReplicaFailoverTest, TestJsonFacets, TestManagedSchema, ConfigureRecoveryStrategyTest, NotRequiredUniqueKeyTest, MinimalSchemaTest, PreAnalyzedFieldTest, BlobRepositoryCloudTest, TestSQLHandler, TestConfigSetsAPIExclusivity, TestNonDefinedSimilarityFactory, SortByFunctionTest, OverseerCollectionConfigSetProcessorTest, TestLFUCache, HLLUtilTest, TestBlobHandler, ConfigSetsAPITest, BigEndianAscendingWordDeserializerTest, FullSolrCloudDistribCmdsTest, SolrMetricReporterTest, CdcrReplicationDistributedZkTest, LeaderFailureAfterFreshStartTest, BasicDistributedZk2Test, DistributedSuggestComponentTest, TestAddFieldRealTimeGet, TestPrepRecovery, TestSolrConfigHandler, DateFieldTest, TestExpandComponent, WrapperMergePolicyFactoryTest, TestFastWriter, AutoCommitTest, TestExclusionRuleCollectionAccess, TestIndexSearcher, TestLegacyFieldReuse, TestSolrCoreSnapshots, CustomCollectionTest, UnloadDistributedZkTest, AnalyticsQueryTest, TestQueryWrapperFilter, AddSchemaFieldsUpdateProcessorFactoryTest, DirectUpdateHandlerTest, TestQuerySenderListener, DateMathParserTest, ShowFileRequestHandlerTest, TestSerializedLuceneMatchVersion, HdfsRecoverLeaseTest, ExplicitHLLTest, SignatureUpdateProcessorFactoryTest, PolyFieldTest, TestReloadAndDeleteDocs, ImplicitSnitchTest, SyncSliceTest, ShardRoutingTest, RecoveryZkTest, TestReplicationHandler, ZkSolrClientTest, DistributedSpellCheckComponentTest, ZkControllerTest, TestStressVersions, HardAutoCommitTest, SimpleFacetsTest, SpellCheckComponentTest, TestLazyCores, SpellCheckCollatorTest, TestCSVLoader, SolrCoreCheckLockOnStartupTest, FieldMutatingUpdateProcessorTest, TestWordDelimiterFilterFactory, StatelessScriptUpdateProcessorFactoryTest, TestSurroundQueryParser, TestWriterPerf, TestQueryTypes, TestOmitPositions, DocumentBuilderTest, PathHierarchyTokenizerFactoryTest, RequiredFieldsTest, FastVectorHighlighterTest, TestJmxIntegration]
   [junit4] Completed [433/729 (2!)] on J2 in 0.77s, 3 tests, 1 error, 1 skipped <<< FAILURES!

[...truncated 37472 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+181) - Build # 20295 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/20295/
Java: 64bit/jdk-9-ea+181 -XX:-UseCompressedOops -XX:+UseParallelGC --illegal-access=deny

1 tests failed.
FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
Doc with id=1 not found in http://127.0.0.1:45957/zu_/gw/collMinRf_1x3 due to: Path not found: /id; rsp={doc=null}

Stack Trace:
java.lang.AssertionError: Doc with id=1 not found in http://127.0.0.1:45957/zu_/gw/collMinRf_1x3 due to: Path not found: /id; rsp={doc=null}
        at __randomizedtesting.SeedInfo.seed([892434F3DB4A5409:1700B2975B639F1]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:603)
        at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:558)
        at org.apache.solr.cloud.HttpPartitionTest.testMinRf(HttpPartitionTest.java:249)
        at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:127)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 11372 lines...]
   [junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/init-core-data-001
   [junit4]   2> 684985 WARN  (SUITE-HttpPartitionTest-seed#[892434F3DB4A5409]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=10 numCloses=10
   [junit4]   2> 684985 INFO  (SUITE-HttpPartitionTest-seed#[892434F3DB4A5409]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 684986 INFO  (SUITE-HttpPartitionTest-seed#[892434F3DB4A5409]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl="https://issues.apache.org/jira/browse/SOLR-5776")
   [junit4]   2> 684986 INFO  (SUITE-HttpPartitionTest-seed#[892434F3DB4A5409]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 684986 INFO  (SUITE-HttpPartitionTest-seed#[892434F3DB4A5409]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /zu_/gw
   [junit4]   2> 684987 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 684987 INFO  (Thread-1222) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 684987 INFO  (Thread-1222) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 684988 ERROR (Thread-1222) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 685087 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.ZkTestServer start zk server on port:41991
   [junit4]   2> 685093 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 685094 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 685095 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 685096 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 685097 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 685097 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 685098 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 685099 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 685112 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 685113 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 685113 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 685114 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
   [junit4]   2> 685176 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 685177 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5b4ce42b{/zu_/gw,null,AVAILABLE}
   [junit4]   2> 685178 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@78a643eb{HTTP/1.1,[http/1.1]}{127.0.0.1:36501}
   [junit4]   2> 685178 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.Server Started @686357ms
   [junit4]   2> 685179 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/zu_/gw, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/tempDir-001/control/data, hostPort=43157, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/control-001/cores, replicaType=NRT}
   [junit4]   2> 685179 ERROR (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 685179 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 685179 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 685179 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 685179 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-11T01:03:17.811351Z
   [junit4]   2> 685180 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 685180 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/control-001/solr.xml
   [junit4]   2> 685182 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 685184 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41991/solr
   [junit4]   2> 685242 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 685243 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43157_zu_%2Fgw
   [junit4]   2> 685243 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.Overseer Overseer (id=98462164426424324-127.0.0.1:43157_zu_%2Fgw-n_0000000000) starting
   [junit4]   2> 685252 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43157_zu_%2Fgw
   [junit4]   2> 685253 INFO  (zkCallback-2535-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 685284 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 685288 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 685288 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 685290 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/control-001/cores
   [junit4]   2> 685301 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 685301 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41991/solr ready
   [junit4]   2> 685301 INFO  (SocketProxy-Acceptor-43157) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=45588,localport=43157], receiveBufferSize:531000
   [junit4]   2> 685311 INFO  (SocketProxy-Acceptor-43157) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=36501,localport=36290], receiveBufferSize=530904
   [junit4]   2> 685312 INFO  (qtp1445921533-11562) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:43157_zu_%252Fgw&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 685314 INFO  (OverseerThreadFactory-3001-thread-1) [    ] o.a.s.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 685417 INFO  (SocketProxy-Acceptor-43157) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=45592,localport=43157], receiveBufferSize:531000
   [junit4]   2> 685418 INFO  (SocketProxy-Acceptor-43157) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=36501,localport=36294], receiveBufferSize=530904
   [junit4]   2> 685418 INFO  (qtp1445921533-11564) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 685418 INFO  (qtp1445921533-11564) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 685523 INFO  (zkCallback-2535-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 686431 INFO  (qtp1445921533-11564) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 686443 INFO  (qtp1445921533-11564) [    ] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 686530 INFO  (qtp1445921533-11564) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 686539 INFO  (qtp1445921533-11564) [    ] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 686539 INFO  (qtp1445921533-11564) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 686539 INFO  (qtp1445921533-11564) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 686539 INFO  (qtp1445921533-11564) [    ] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 686541 INFO  (qtp1445921533-11564) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=47.2880859375, floorSegmentMB=0.8759765625, forceMergeDeletesPctAllowed=17.995157588268476, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.39833529615685725
   [junit4]   2> 686545 WARN  (qtp1445921533-11564) [    ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 686570 INFO  (qtp1445921533-11564) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 686570 INFO  (qtp1445921533-11564) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 686571 INFO  (qtp1445921533-11564) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 686571 INFO  (qtp1445921533-11564) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 686571 INFO  (qtp1445921533-11564) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=65.4072265625, floorSegmentMB=1.169921875, forceMergeDeletesPctAllowed=14.716761857966459, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.16305439323964804
   [junit4]   2> 686571 INFO  (qtp1445921533-11564) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@586bb3f8[control_collection_shard1_replica_n1] main]
   [junit4]   2> 686572 INFO  (qtp1445921533-11564) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 686572 INFO  (qtp1445921533-11564) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 686572 INFO  (qtp1445921533-11564) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 686573 INFO  (searcherExecutor-3004-thread-1) [    ] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@586bb3f8[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 686573 INFO  (qtp1445921533-11564) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575394632484782080
   [junit4]   2> 686576 INFO  (qtp1445921533-11564) [    ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 686576 INFO  (qtp1445921533-11564) [    ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 686576 INFO  (qtp1445921533-11564) [    ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43157/zu_/gw/control_collection_shard1_replica_n1/
   [junit4]   2> 686576 INFO  (qtp1445921533-11564) [    ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 686576 INFO  (qtp1445921533-11564) [    ] o.a.s.c.SyncStrategy http://127.0.0.1:43157/zu_/gw/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 686576 INFO  (qtp1445921533-11564) [    ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 686581 INFO  (qtp1445921533-11564) [    ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43157/zu_/gw/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 686683 INFO  (zkCallback-2535-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 686732 INFO  (qtp1445921533-11564) [    ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 686733 INFO  (qtp1445921533-11564) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1315
   [junit4]   2> 686736 INFO  (qtp1445921533-11562) [    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 686836 INFO  (zkCallback-2535-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 687315 INFO  (OverseerCollectionConfigSetProcessor-98462164426424324-127.0.0.1:43157_zu_%2Fgw-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 687736 INFO  (qtp1445921533-11562) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:43157_zu_%252Fgw&wt=javabin&version=2} status=0 QTime=2423
   [junit4]   2> 687739 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 687739 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41991/solr ready
   [junit4]   2> 687739 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 687740 INFO  (SocketProxy-Acceptor-43157) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=45612,localport=43157], receiveBufferSize:531000
   [junit4]   2> 687740 INFO  (qtp1445921533-11559) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 687741 INFO  (SocketProxy-Acceptor-43157) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=36501,localport=36314], receiveBufferSize=530904
   [junit4]   2> 687744 INFO  (OverseerThreadFactory-3001-thread-2) [    ] o.a.s.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 687744 WARN  (OverseerThreadFactory-3001-thread-2) [    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 687947 INFO  (qtp1445921533-11559) [    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 687947 INFO  (qtp1445921533-11559) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=207
   [junit4]   2> 688061 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-1-001 of type TLOG
   [junit4]   2> 688062 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 688063 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@e20b99e{/zu_/gw,null,AVAILABLE}
   [junit4]   2> 688063 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@23147fb3{HTTP/1.1,[http/1.1]}{127.0.0.1:39619}
   [junit4]   2> 688063 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.Server Started @689242ms
   [junit4]   2> 688063 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/zu_/gw, solrconfig=solrconfig.xml, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/tempDir-001/jetty1, hostPort=41969, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-1-001/cores, replicaType=TLOG}
   [junit4]   2> 688064 ERROR (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 688064 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 688064 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 688064 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 688064 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-11T01:03:20.696502Z
   [junit4]   2> 688066 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 688066 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-1-001/solr.xml
   [junit4]   2> 688086 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 688089 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41991/solr
   [junit4]   2> 688097 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 688098 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 688099 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41969_zu_%2Fgw
   [junit4]   2> 688100 INFO  (zkCallback-2535-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 688100 INFO  (zkCallback-2542-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 688100 INFO  (zkCallback-2547-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 688146 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 688152 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 688152 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 688154 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-1-001/cores
   [junit4]   2> 688171 INFO  (qtp1445921533-11563) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:41969_zu_%252Fgw&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 688173 INFO  (OverseerCollectionConfigSetProcessor-98462164426424324-127.0.0.1:43157_zu_%2Fgw-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 688173 INFO  (OverseerThreadFactory-3001-thread-3) [    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:41969_zu_%2Fgw for creating new replica
   [junit4]   2> 688174 INFO  (SocketProxy-Acceptor-41969) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44502,localport=41969], receiveBufferSize:531000
   [junit4]   2> 688174 INFO  (SocketProxy-Acceptor-41969) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39619,localport=58088], receiveBufferSize=530904
   [junit4]   2> 688175 INFO  (qtp600471921-11611) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 688175 INFO  (qtp600471921-11611) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 688277 INFO  (zkCallback-2547-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 689184 INFO  (qtp600471921-11611) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 689193 INFO  (qtp600471921-11611) [    ] o.a.s.s.IndexSchema [collection1_shard2_replica_t41] Schema name=test
   [junit4]   2> 689636 INFO  (qtp600471921-11611) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 689644 INFO  (qtp600471921-11611) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t41' using configuration from collection collection1, trusted=true
   [junit4]   2> 689647 INFO  (qtp600471921-11611) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t41' (registry 'solr.core.collection1.shard2.replica_t41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 689647 INFO  (qtp600471921-11611) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 689647 INFO  (qtp600471921-11611) [    ] o.a.s.c.SolrCore [[collection1_shard2_replica_t41] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-1-001/cores/collection1_shard2_replica_t41], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-1-001/cores/collection1_shard2_replica_t41/data/]
   [junit4]   2> 689649 INFO  (qtp600471921-11611) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=47.2880859375, floorSegmentMB=0.8759765625, forceMergeDeletesPctAllowed=17.995157588268476, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.39833529615685725
   [junit4]   2> 689652 WARN  (qtp600471921-11611) [    ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 689699 INFO  (qtp600471921-11611) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 689699 INFO  (qtp600471921-11611) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 689700 INFO  (qtp600471921-11611) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 689700 INFO  (qtp600471921-11611) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 689701 INFO  (qtp600471921-11611) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=65.4072265625, floorSegmentMB=1.169921875, forceMergeDeletesPctAllowed=14.716761857966459, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.16305439323964804
   [junit4]   2> 689702 INFO  (qtp600471921-11611) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@53a34d94[collection1_shard2_replica_t41] main]
   [junit4]   2> 689702 INFO  (qtp600471921-11611) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 689702 INFO  (qtp600471921-11611) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 689703 INFO  (qtp600471921-11611) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 689703 INFO  (qtp600471921-11611) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575394635766824960
   [junit4]   2> 689704 INFO  (searcherExecutor-3015-thread-1) [    ] o.a.s.c.SolrCore [collection1_shard2_replica_t41] Registered new searcher Searcher@53a34d94[collection1_shard2_replica_t41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 689706 INFO  (qtp600471921-11611) [    ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 689707 INFO  (qtp600471921-11611) [    ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 689707 INFO  (qtp600471921-11611) [    ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41969/zu_/gw/collection1_shard2_replica_t41/
   [junit4]   2> 689707 INFO  (qtp600471921-11611) [    ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 689707 INFO  (qtp600471921-11611) [    ] o.a.s.c.SyncStrategy http://127.0.0.1:41969/zu_/gw/collection1_shard2_replica_t41/ has no replicas
   [junit4]   2> 689707 INFO  (qtp600471921-11611) [    ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 689707 INFO  (qtp600471921-11611) [    ] o.a.s.c.ZkController collection1_shard2_replica_t41 stopping background replication from leader
   [junit4]   2> 689708 INFO  (qtp600471921-11611) [    ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41969/zu_/gw/collection1_shard2_replica_t41/ shard2
   [junit4]   2> 689810 INFO  (zkCallback-2547-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 689859 INFO  (qtp600471921-11611) [    ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 689860 INFO  (qtp600471921-11611) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1685
   [junit4]   2> 689862 INFO  (qtp1445921533-11563) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:41969_zu_%252Fgw&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2} status=0 QTime=1690
   [junit4]   2> 689928 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-2-001 of type TLOG
   [junit4]   2> 689929 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 689929 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5a000ab1{/zu_/gw,null,AVAILABLE}
   [junit4]   2> 689930 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1515f805{HTTP/1.1,[http/1.1]}{127.0.0.1:44625}
   [junit4]   2> 689930 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.Server Started @691109ms
   [junit4]   2> 689930 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/zu_/gw, solrconfig=solrconfig.xml, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/tempDir-001/jetty2, hostPort=45957, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-2-001/cores, replicaType=TLOG}
   [junit4]   2> 689930 ERROR (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 689930 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 689930 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 689930 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 689930 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-11T01:03:22.562798Z
   [junit4]   2> 689932 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 689932 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-2-001/solr.xml
   [junit4]   2> 689942 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 689944 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41991/solr
   [junit4]   2> 689948 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 689949 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 689950 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45957_zu_%2Fgw
   [junit4]   2> 689950 INFO  (zkCallback-2542-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 689950 INFO  (zkCallback-2547-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 689950 INFO  (zkCallback-2535-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 689953 INFO  (zkCallback-2553-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 690007 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 690013 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 690013 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 690014 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-2-001/cores
   [junit4]   2> 690038 INFO  (qtp1445921533-11564) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:45957_zu_%252Fgw&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 690039 INFO  (OverseerCollectionConfigSetProcessor-98462164426424324-127.0.0.1:43157_zu_%2Fgw-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 690039 INFO  (OverseerThreadFactory-3001-thread-4) [    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:45957_zu_%2Fgw for creating new replica
   [junit4]   2> 690040 INFO  (SocketProxy-Acceptor-45957) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=37436,localport=45957], receiveBufferSize:531000
   [junit4]   2> 690041 INFO  (SocketProxy-Acceptor-45957) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=44625,localport=40458], receiveBufferSize=530904
   [junit4]   2> 690042 INFO  (qtp36373676-11643) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 690042 INFO  (qtp36373676-11643) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 690144 INFO  (zkCallback-2547-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 690144 INFO  (zkCallback-2553-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 691052 INFO  (qtp36373676-11643) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 691064 INFO  (qtp36373676-11643) [    ] o.a.s.s.IndexSchema [collection1_shard1_replica_t43] Schema name=test
   [junit4]   2> 691192 INFO  (qtp36373676-11643) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 691200 INFO  (qtp36373676-11643) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t43' using configuration from collection collection1, trusted=true
   [junit4]   2> 691200 INFO  (qtp36373676-11643) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_t43' (registry 'solr.core.collection1.shard1.replica_t43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 691200 INFO  (qtp36373676-11643) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 691201 INFO  (qtp36373676-11643) [    ] o.a.s.c.SolrCore [[collection1_shard1_replica_t43] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-2-001/cores/collection1_shard1_replica_t43], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-2-001/cores/collection1_shard1_replica_t43/data/]
   [junit4]   2> 691203 INFO  (qtp36373676-11643) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=47.2880859375, floorSegmentMB=0.8759765625, forceMergeDeletesPctAllowed=17.995157588268476, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.39833529615685725
   [junit4]   2> 691205 WARN  (qtp36373676-11643) [    ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 691241 INFO  (qtp36373676-11643) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 691241 INFO  (qtp36373676-11643) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 691242 INFO  (qtp36373676-11643) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 691242 INFO  (qtp36373676-11643) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 691243 INFO  (qtp36373676-11643) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=65.4072265625, floorSegmentMB=1.169921875, forceMergeDeletesPctAllowed=14.716761857966459, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.16305439323964804
   [junit4]   2> 691243 INFO  (qtp36373676-11643) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@33c22912[collection1_shard1_replica_t43] main]
   [junit4]   2> 691244 INFO  (qtp36373676-11643) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 691244 INFO  (qtp36373676-11643) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 691245 INFO  (qtp36373676-11643) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 691246 INFO  (searcherExecutor-3026-thread-1) [    ] o.a.s.c.SolrCore [collection1_shard1_replica_t43] Registered new searcher Searcher@33c22912[collection1_shard1_replica_t43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 691246 INFO  (qtp36373676-11643) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575394637384777728
   [junit4]   2> 691252 INFO  (qtp36373676-11643) [    ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 691253 INFO  (qtp36373676-11643) [    ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 691253 INFO  (qtp36373676-11643) [    ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45957/zu_/gw/collection1_shard1_replica_t43/
   [junit4]   2> 691253 INFO  (qtp36373676-11643) [    ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 691253 INFO  (qtp36373676-11643) [    ] o.a.s.c.SyncStrategy http://127.0.0.1:45957/zu_/gw/collection1_shard1_replica_t43/ has no replicas
   [junit4]   2> 691253 INFO  (qtp36373676-11643) [    ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 691253 INFO  (qtp36373676-11643) [    ] o.a.s.c.ZkController collection1_shard1_replica_t43 stopping background replication from leader
   [junit4]   2> 691255 INFO  (qtp36373676-11643) [    ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45957/zu_/gw/collection1_shard1_replica_t43/ shard1
   [junit4]   2> 691356 INFO  (zkCallback-2547-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 691356 INFO  (zkCallback-2553-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 691362 INFO  (qtp36373676-11643) [    ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 691363 INFO  (qtp36373676-11643) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1321
   [junit4]   2> 691365 INFO  (qtp1445921533-11564) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:45957_zu_%252Fgw&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2} status=0 QTime=1327
   [junit4]   2> 691431 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-3-001 of type TLOG
   [junit4]   2> 691431 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 691432 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2cca6a5d{/zu_/gw,null,AVAILABLE}
   [junit4]   2> 691432 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1577f773{HTTP/1.1,[http/1.1]}{127.0.0.1:42585}
   [junit4]   2> 691433 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.Server Started @692611ms
   [junit4]   2> 691433 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/zu_/gw, solrconfig=solrconfig.xml, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/tempDir-001/jetty3, hostPort=45487, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-3-001/cores, replicaType=TLOG}
   [junit4]   2> 691433 ERROR (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 691433 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 691433 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 691433 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 691433 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-11T01:03:24.065557Z
   [junit4]   2> 691434 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 691434 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-3-001/solr.xml
   [junit4]   2> 691447 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 691448 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41991/solr
   [junit4]   2> 691452 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 691453 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 691454 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45487_zu_%2Fgw
   [junit4]   2> 691454 INFO  (zkCallback-2542-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 691454 INFO  (zkCallback-2547-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 691454 INFO  (zkCallback-2553-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 691456 INFO  (zkCallback-2535-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 691456 INFO  (zkCallback-2559-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 691487 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 691490 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 691490 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 691491 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-3-001/cores
   [junit4]   2> 691510 INFO  (qtp1445921533-11562) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:45487_zu_%252Fgw&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 691511 INFO  (OverseerCollectionConfigSetProcessor-98462164426424324-127.0.0.1:43157_zu_%2Fgw-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 691511 INFO  (OverseerThreadFactory-3001-thread-5) [    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:45487_zu_%2Fgw for creating new replica
   [junit4]   2> 691512 INFO  (SocketProxy-Acceptor-45487) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=33208,localport=45487], receiveBufferSize:531000
   [junit4]   2> 691512 INFO  (SocketProxy-Acceptor-45487) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=42585,localport=53582], receiveBufferSize=530904
   [junit4]   2> 691512 INFO  (qtp1558833314-11675) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 691512 INFO  (qtp1558833314-11675) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 691615 INFO  (zkCallback-2547-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 691615 INFO  (zkCallback-2553-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 691615 INFO  (zkCallback-2559-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 692522 INFO  (qtp1558833314-11675) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 692532 INFO  (qtp1558833314-11675) [    ] o.a.s.s.IndexSchema [collection1_shard2_replica_t45] Schema name=test
   [junit4]   2> 692619 INFO  (qtp1558833314-11675) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 692625 INFO  (qtp1558833314-11675) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t45' using configuration from collection collection1, trusted=true
   [junit4]   2> 692625 INFO  (qtp1558833314-11675) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t45' (registry 'solr.core.collection1.shard2.replica_t45') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 692625 INFO  (qtp1558833314-11675) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 692625 INFO  (qtp1558833314-11675) [    ] o.a.s.c.SolrCore [[collection1_shard2_replica_t45] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-3-001/cores/collection1_shard2_replica_t45], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/shard-3-001/cores/collection1_shard2_replica_t45/data/]
   [junit4]   2> 692627 INFO  (qtp1558833314-11675) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=47.2880859375, floorSegmentMB=0.8759765625, forceMergeDeletesPctAllowed=17.995157588268476, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.39833529615685725
   [junit4]   2> 692628 WARN  (qtp1558833314-11675) [    ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 692650 INFO  (qtp1558833314-11675) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 692651 INFO  (qtp1558833314-11675) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 692651 INFO  (qtp1558833314-11675) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 692651 INFO  (qtp1558833314-11675) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 692652 INFO  (qtp1558833314-11675) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=65.4072265625, floorSegmentMB=1.169921875, forceMergeDeletesPctAllowed=14.716761857966459, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.16305439323964804
   [junit4]   2> 692652 INFO  (qtp1558833314-11675) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@70047bfd[collection1_shard2_replica_t45] main]
   [junit4]   2> 692652 INFO  (qtp1558833314-11675) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 692653 INFO  (qtp1558833314-11675) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 692653 INFO  (qtp1558833314-11675) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 692654 INFO  (searcherExecutor-3037-thread-1) [    ] o.a.s.c.SolrCore [collection1_shard2_replica_t45] Registered new searcher Searcher@70047bfd[collection1_shard2_replica_t45] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 692654 INFO  (qtp1558833314-11675) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575394638861172736
   [junit4]   2> 692656 INFO  (qtp1558833314-11675) [    ] o.a.s.c.ZkController Core needs to recover:collection1_shard2_replica_t45
   [junit4]   2> 692656 INFO  (updateExecutor-2556-thread-1) [    ] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 692656 INFO  (qtp1558833314-11675) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1143
   [junit4]   2> 692656 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 692656 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 692656 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.c.ZkController collection1_shard2_replica_t45 stopping background replication from leader
   [junit4]   2> 692656 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard2_replica_t45]
   [junit4]   2> 692656 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 692656 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard2_replica_t45] as recovering, leader is [http://127.0.0.1:41969/zu_/gw/collection1_shard2_replica_t41/] and I am [http://127.0.0.1:45487/zu_/gw/collection1_shard2_replica_t45/]
   [junit4]   2> 692657 INFO  (qtp1445921533-11562) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:45487_zu_%252Fgw&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2} status=0 QTime=1147
   [junit4]   2> 692658 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:41969/zu_/gw]; [WaitForState: action=PREPRECOVERY&core=collection1_shard2_replica_t41&nodeName=127.0.0.1:45487_zu_%252Fgw&coreNodeName=core_node46&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 692658 INFO  (SocketProxy-Acceptor-41969) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44560,localport=41969], receiveBufferSize:531000
   [junit4]   2> 692658 INFO  (SocketProxy-Acceptor-41969) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39619,localport=58146], receiveBufferSize=530904
   [junit4]   2> 692659 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 692660 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Wait for recoveries to finish - wait 30000 for each attempt
   [junit4]   2> 692660 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):30000
   [junit4]   2> 692660 INFO  (qtp600471921-11613) [    ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node46, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
   [junit4]   2> 692660 INFO  (qtp600471921-11613) [    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1_shard2_replica_t41, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:45487_zu_%2Fgw, coreNodeName=core_node46, onlyIfActiveCheckResult=false, nodeProps: core_node46:{"core":"collection1_shard2_replica_t45","base_url":"http://127.0.0.1:45487/zu_/gw","node_name":"127.0.0.1:45487_zu_%2Fgw","state":"down","type":"TLOG"}
   [junit4]   2> 692759 INFO  (zkCallback-2559-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 692759 INFO  (zkCallback-2547-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 692759 INFO  (zkCallback-2553-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 693512 INFO  (OverseerCollectionConfigSetProcessor-98462164426424324-127.0.0.1:43157_zu_%2Fgw-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 693660 INFO  (qtp600471921-11613) [    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1_shard2_replica_t41, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:45487_zu_%2Fgw, coreNodeName=core_node46, onlyIfActiveCheckResult=false, nodeProps: core_node46:{"core":"collection1_shard2_replica_t45","base_url":"http://127.0.0.1:45487/zu_/gw","node_name":"127.0.0.1:45487_zu_%2Fgw","state":"recovering","type":"TLOG"}
   [junit4]   2> 693660 INFO  (qtp600471921-11613) [    ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node46, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
   [junit4]   2> 693660 INFO  (qtp600471921-11613) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:45487_zu_%252Fgw&onlyIfLeaderActive=true&core=collection1_shard2_replica_t41&coreNodeName=core_node46&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1000
   [junit4]   2> 694161 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.c.RecoveryStrategy Starting Replication Recovery.
   [junit4]   2> 694161 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.c.RecoveryStrategy Attempting to replicate from [http://127.0.0.1:41969/zu_/gw/collection1_shard2_replica_t41/].
   [junit4]   2> 694162 INFO  (SocketProxy-Acceptor-41969) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44574,localport=41969], receiveBufferSize:531000
   [junit4]   2> 694162 INFO  (SocketProxy-Acceptor-41969) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39619,localport=58160], receiveBufferSize=530904
   [junit4]   2> 694163 INFO  (qtp600471921-11613) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1575394640443473920,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 694163 INFO  (qtp600471921-11613) [    ] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 694163 INFO  (qtp600471921-11613) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 694163 INFO  (qtp600471921-11613) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_t41]  webapp=/zu_/gw path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2}{commit=} 0 0
   [junit4]   2> 694164 INFO  (SocketProxy-Acceptor-41969) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44578,localport=41969], receiveBufferSize:531000
   [junit4]   2> 694164 INFO  (SocketProxy-Acceptor-41969) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39619,localport=58164], receiveBufferSize=530904
   [junit4]   2> 694165 INFO  (qtp600471921-11609) [    ] o.a.s.c.S.Request [collection1_shard2_replica_t41]  webapp=/zu_/gw path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 694165 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 694165 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 694165 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 694165 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 694165 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.c.RecoveryStrategy Replication Recovery was successful.
   [junit4]   2> 694165 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
   [junit4]   2> 694165 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.c.ZkController collection1_shard2_replica_t45 starting background replication from leader
   [junit4]   2> 694165 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
   [junit4]   2> 694166 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
   [junit4]   2> 694166 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.c.RecoveryStrategy Updating version bucket highest from index after successful recovery.
   [junit4]   2> 694166 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575394640446619648
   [junit4]   2> 694168 INFO  (recoveryExecutor-2557-thread-1) [    ] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[true]
   [junit4]   2> 694268 INFO  (zkCallback-2547-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 694268 INFO  (zkCallback-2559-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 694268 INFO  (zkCallback-2553-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 694339 INFO  (indexFetcher-3043-thread-1) [    ] o.a.s.h.IndexFetcher Updated masterUrl to http://127.0.0.1:41969/zu_/gw/collection1_shard2_replica_t41/
   [junit4]   2> 694340 INFO  (qtp600471921-11608) [    ] o.a.s.c.S.Request [collection1_shard2_replica_t41]  webapp=/zu_/gw path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 694340 INFO  (indexFetcher-3043-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 694340 INFO  (indexFetcher-3043-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 694340 INFO  (indexFetcher-3043-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 694340 INFO  (indexFetcher-3043-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 694660 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
   [junit4]   2> 694662 INFO  (qtp1445921533-11563) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1575394640966713344,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 694662 INFO  (qtp1445921533-11563) [    ] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 694662 INFO  (qtp1445921533-11563) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 694662 INFO  (qtp1445921533-11563) [    ] o.a.s.u.p.LogUpdateProcessorFactory [control_collection_shard1_replica_n1]  webapp=/zu_/gw path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 0
   [junit4]   2> 694663 INFO  (SocketProxy-Acceptor-45957) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=37500,localport=45957], receiveBufferSize:531000
   [junit4]   2> 694664 INFO  (SocketProxy-Acceptor-45957) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=44625,localport=40522], receiveBufferSize=530904
   [junit4]   2> 694673 INFO  (SocketProxy-Acceptor-45957) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=37504,localport=45957], receiveBufferSize:531000
   [junit4]   2> 694680 INFO  (SocketProxy-Acceptor-45957) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=44625,localport=40526], receiveBufferSize=530904
   [junit4]   2> 694682 INFO  (SocketProxy-Acceptor-41969) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44598,localport=41969], receiveBufferSize:531000
   [junit4]   2> 694683 INFO  (SocketProxy-Acceptor-41969) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39619,localport=58184], receiveBufferSize=530904
   [junit4]   2> 694683 INFO  (qtp36373676-11640) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1575394640988733440,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 694683 INFO  (qtp36373676-11640) [    ] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 694683 INFO  (qtp36373676-11640) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 694683 INFO  (SocketProxy-Acceptor-45487) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=33262,localport=45487], receiveBufferSize:531000
   [junit4]   2> 694684 INFO  (qtp36373676-11640) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_t43]  webapp=/zu_/gw path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:45957/zu_/gw/collection1_shard1_replica_t43/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 1
   [junit4]   2> 694684 INFO  (SocketProxy-Acceptor-45487) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=42585,localport=53636], receiveBufferSize=530904
   [junit4]   2> 694684 INFO  (qtp600471921-11607) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1575394640989782016,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 694684 INFO  (qtp1558833314-11677) [    ] o.a.s.u.TestInjection Start waiting for replica in sync with leader
   [junit4]   2> 694684 INFO  (qtp600471921-11607) [    ] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 694684 INFO  (qtp600471921-11607) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 694685 INFO  (qtp600471921-11607) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_t41]  webapp=/zu_/gw path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:45957/zu_/gw/collection1_shard1_replica_t43/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
   [junit4]   2> 694689 INFO  (SocketProxy-Acceptor-41969) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44606,localport=41969], receiveBufferSize:531000
   [junit4]   2> 694693 INFO  (qtp600471921-11608) [    ] o.a.s.c.S.Request [collection1_shard2_replica_t41]  webapp=/zu_/gw path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
   [junit4]   2> 694693 INFO  (SocketProxy-Acceptor-41969) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39619,localport=58192], receiveBufferSize=530904
   [junit4]   2> 694693 INFO  (qtp1558833314-11677) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_t45]  webapp=/zu_/gw path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:45957/zu_/gw/collection1_shard1_replica_t43/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 9
   [junit4]   2> 694694 INFO  (qtp36373676-11639) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_t43]  webapp=/zu_/gw path=/update params={_stateVer_=collection1:7&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 25
   [junit4]   2> 694696 INFO  (SocketProxy-Acceptor-41969) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44610,localport=41969], receiveBufferSize:531000
   [junit4]   2> 694698 INFO  (SocketProxy-Acceptor-41969) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39619,localport=58196], receiveBufferSize=530904
   [junit4]   2> 694698 INFO  (qtp600471921-11609) [    ] o.a.s.c.S.Request [collection1_shard2_replica_t41]  webapp=/zu_/gw path=/select params={q=*:*&distrib=false&tests=checkShardConsistency&rows=0&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 694699 INFO  (SocketProxy-Acceptor-45487) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=33274,localport=45487], receiveBufferSize:531000
   [junit4]   2> 694702 INFO  (qtp1558833314-11672) [    ] o.a.s.c.S.Request [collection1_shard2_replica_t45]  webapp=/zu_/gw path=/select params={q=*:*&distrib=false&tests=checkShardConsistency&rows=0&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 694702 INFO  (SocketProxy-Acceptor-45487) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=42585,localport=53648], receiveBufferSize=530904
   [junit4]   2> 694703 INFO  (SocketProxy-Acceptor-45957) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=37528,localport=45957], receiveBufferSize:531000
   [junit4]   2> 694704 INFO  (SocketProxy-Acceptor-45957) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=44625,localport=40550], receiveBufferSize=530904
   [junit4]   2> 694705 INFO  (qtp36373676-11638) [    ] o.a.s.c.S.Request [collection1_shard1_replica_t43]  webapp=/zu_/gw path=/select params={q=*:*&distrib=false&tests=checkShardConsistency&rows=0&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 696716 INFO  (SocketProxy-Acceptor-41969) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44634,localport=41969], receiveBufferSize:531000
   [junit4]   2> 696717 INFO  (qtp600471921-11611) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&maxShardsPerNode=1&collection.configName=conf1&nrtReplicas=0&name=c8n_crud_1x2&action=CREATE&numShards=1&tlogReplicas=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 696719 INFO  (OverseerThreadFactory-3001-thread-5) [    ] o.a.s.c.CreateCollectionCmd Create collection c8n_crud_1x2
   [junit4]   2> 696720 INFO  (SocketProxy-Acceptor-41969) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39619,localport=58220], receiveBufferSize=530904
   [junit4]   2> 696826 INFO  (qtp1445921533-11557) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=c8n_crud_1x2_shard1_replica_t1&action=CREATE&numShards=1&collection=c8n_crud_1x2&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 696827 INFO  (qtp600471921-11612) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=c8n_crud_1x2_shard1_replica_t2&action=CREATE&numShards=1&collection=c8n_crud_1x2&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 696930 INFO  (zkCallback-2547-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/c8n_crud_1x2/state.json] for collection [c8n_crud_1x2] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 696930 INFO  (zkCallback-2535-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/c8n_crud_1x2/state.json] for collection [c8n_crud_1x2] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 697341 INFO  (qtp600471921-11613) [    ] o.a.s.c.S.Request [collection1_shard2_replica_t41]  webapp=/zu_/gw path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 697341 INFO  (indexFetcher-3043-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 697341 INFO  (indexFetcher-3043-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 697341 INFO  (indexFetcher-3043-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 697341 INFO  (indexFetcher-3043-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 697834 INFO  (qtp600471921-11612) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 697839 INFO  (qtp1445921533-11557) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 697853 INFO  (qtp600471921-11612) [    ] o.a.s.s.IndexSchema [c8n_crud_1x2_shard1_replica_t2] Schema name=test
   [junit4]   2> 697854 INFO  (qtp1445921533-11557) [    ] o.a.s.s.IndexSchema [c8n_crud_1x2_shard1_replica_t1] Schema name=test
   [junit4]   2> 697940 INFO  (qtp1445921533-11557) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 697953 INFO  (qtp1445921533-11557) [    ] o.a.s.c.CoreContainer Creating SolrCore 'c8n_crud_1x2_shard1_replica_t1' using configuration from collection c8n_crud_1x2, trusted=true
   [junit4]   2> 697953 INFO  (qtp1445921533-11557) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.c8n_crud_1x2.shard1.replica_t1' (registry 'solr.core.c8n_crud_1x2.shard1.replica_t1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6bba6964
   [junit4]   2> 697954 INFO  (qtp1445921533-11557) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 697954 INFO  (qtp1445921533-11557) [    ] o.a.s.c.SolrCore [[c8n_crud_1x2_shard1_replica_t1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/control-001/cores/c8n_crud_1x2_shard1_replica_t1], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001/control-001/cores/c8n_crud_1x2_shard1_replica_t1/data/]
   [junit4]   2> 697955 INFO  (qtp600471921-11612) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 697955 INFO  (qtp1445921533-11557) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=47.2880859375, floorSegmentMB=0.8759765625, forceMergeDeletesPctAllowed=17.995157588268476, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.39833529615685725
   [junit4]   2> 697957 WARN  (qtp1445921533-11557) [    ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 697962 INFO  (qtp600471921-11612) [    ] o.a.s.c.CoreContainer Creating SolrCore 'c8n_crud_1x2_shard1_replica_t2' using configuration from collection c8n_crud_1x2, trusted=true
   [junit4]   2> 697962 INFO  (qtp600471921-11612) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.c8n_crud_1

[...truncated too long message...]

  2> at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332)
   [junit4]   2> at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)
   [junit4]   2> at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1183)
   [junit4]   2> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514)
   [junit4]   2> at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
   [junit4]   2> at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:300)
   [junit4]   2> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
   [junit4]   2> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4]   2> at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2>
   [junit4]   2> 714407 WARN  (indexFetcher-3043-thread-1) [    ] o.a.s.h.IndexFetcher Could not log failed replication details
   [junit4]   2> org.apache.lucene.store.AlreadyClosedException: Already closed
   [junit4]   2> at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:337)
   [junit4]   2> at org.apache.solr.handler.ReplicationHandler.loadReplicationProperties(ReplicationHandler.java:1131)
   [junit4]   2> at org.apache.solr.handler.IndexFetcher.logReplicationTimeAndConfFiles(IndexFetcher.java:769)
   [junit4]   2> at org.apache.solr.handler.IndexFetcher.cleanup(IndexFetcher.java:677)
   [junit4]   2> at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:659)
   [junit4]   2> at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332)
   [junit4]   2> at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)
   [junit4]   2> at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1183)
   [junit4]   2> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514)
   [junit4]   2> at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
   [junit4]   2> at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:300)
   [junit4]   2> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
   [junit4]   2> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4]   2> at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 715894 WARN  (zkCallback-2559-thread-1) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 715894 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@1577f773{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 715894 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2cca6a5d{/zu_/gw,null,UNAVAILABLE}
   [junit4]   2> 715895 ERROR (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 715895 INFO  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41991 41991
   [junit4]   2> 719752 INFO  (Thread-1222) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41991 41991
   [junit4]   2> 719752 WARN  (Thread-1222) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 6 /solr/aliases.json
   [junit4]   2> 4 /solr/security.json
   [junit4]   2> 4 /solr/configs/conf1
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 6 /solr/clusterprops.json
   [junit4]   2> 6 /solr/clusterstate.json
   [junit4]   2> 3 /solr/collections/collection1/state.json
   [junit4]   2> 3 /solr/collections/collMinRf_1x3/state.json
   [junit4]   2> 2 /solr/collections/c8n_crud_1x2/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 6 /solr/live_nodes
   [junit4]   2> 6 /solr/collections
   [junit4]   2> 4 /solr/overseer/collection-queue-work
   [junit4]   2> 3 /solr/overseer/queue
   [junit4]   2> 2 /solr/overseer/queue-work
   [junit4]   2>
   [junit4]   2> 719753 WARN  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.SocketProxy Closing 3 connections to: http://127.0.0.1:43157/zu_/gw, target: http://127.0.0.1:36501/zu_/gw
   [junit4]   2> 719753 WARN  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.SocketProxy Closing 14 connections to: http://127.0.0.1:41969/zu_/gw, target: http://127.0.0.1:39619/zu_/gw
   [junit4]   2> 719753 WARN  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.SocketProxy Closing 8 connections to: http://127.0.0.1:45957/zu_/gw, target: http://127.0.0.1:44625/zu_/gw
   [junit4]   2> 719753 WARN  (TEST-HttpPartitionTest.test-seed#[892434F3DB4A5409]) [    ] o.a.s.c.SocketProxy Closing 2 connections to: http://127.0.0.1:45487/zu_/gw, target: http://127.0.0.1:42585/zu_/gw
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HttpPartitionTest -Dtests.method=test -Dtests.seed=892434F3DB4A5409 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=fr-CH -Dtests.timezone=America/Guyana -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 34.8s J2 | HttpPartitionTest.test <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Doc with id=1 not found in http://127.0.0.1:45957/zu_/gw/collMinRf_1x3 due to: Path not found: /id; rsp={doc=null}
   [junit4]    > at __randomizedtesting.SeedInfo.seed([892434F3DB4A5409:1700B2975B639F1]:0)
   [junit4]    > at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:603)
   [junit4]    > at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:558)
   [junit4]    > at org.apache.solr.cloud.HttpPartitionTest.testMinRf(HttpPartitionTest.java:249)
   [junit4]    > at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:127)
   [junit4]    > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > at java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
   [junit4]    > at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_892434F3DB4A5409-001
   [junit4]   2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=false): {}, locale=fr-CH, timezone=America/Guyana
   [junit4]   2> NOTE: Linux 4.10.0-27-generic amd64/Oracle Corporation 9 (64-bit)/cpus=8,threads=1,free=234389072,total=528482304
   [junit4]   2> NOTE: All tests run in this JVM: [OverseerStatusTest, TestMultiValuedNumericRangeQuery, TestHighFrequencyDictionaryFactory, TestPseudoReturnFields, BasicDistributedZk2Test, TestExactStatsCache, TestLFUCache, SolrMetricReporterTest, TestSuggestSpellingConverter, TestFastWriter, ResponseLogComponentTest, DistributedExpandComponentTest, SSLMigrationTest, TestCloudNestedDocsSort, TestCollectionAPI, XsltUpdateRequestHandlerTest, TestSolrCloudWithSecureImpersonation, TestStressLiveNodes, TestLeaderElectionZkExpiry, SuggesterFSTTest, LeaderFailureAfterFreshStartTest, TestJavabinTupleStreamParser, ReturnFieldsTest, TestIndexSearcher, UnloadDistributedZkTest, TestSSLRandomization, TestInfoStreamLogging, SolrInfoBeanTest, TestPointFields, TestUseDocValuesAsStored, SmileWriterTest, AsyncCallRequestStatusResponseTest, BufferStoreTest, SimpleMLTQParserTest, DeleteStatusTest, TestUtils, TestSerializedLuceneMatchVersion, TestFilteredDocIdSet, TestSizeLimitedDistributedMap, DocumentBuilderTest, StatsComponentTest, SharedFSAutoReplicaFailoverTest, TestSchemaResource, CoreMergeIndexesAdminHandlerTest, HLLUtilTest, RecoveryAfterSoftCommitTest, DocValuesNotIndexedTest, TestInitQParser, TestSolr4Spatial2, SortByFunctionTest, TermVectorComponentDistributedTest, TestOverriddenPrefixQueryForCustomFieldType, TestManagedSynonymFilterFactory, PathHierarchyTokenizerFactoryTest, PeerSyncWithIndexFingerprintCachingTest, TestTrackingShardHandlerFactory, ClusterStateTest, AnalyticsQueryTest, RecoveryZkTest, TestTolerantSearch, VMParamsZkACLAndCredentialsProvidersTest, TestDefaultStatsCache, TestExportWriter, NumericFieldsTest, ManagedSchemaRoundRobinCloudTest, MetricsConfigTest, TestRecovery, TestBlendedInfixSuggestions, DistributedSuggestComponentTest, TestReversedWildcardFilterFactory, TestAuthenticationFramework, BlobRepositoryCloudTest, ExitableDirectoryReaderTest, TestScoreJoinQPScore, TestDocTermOrds, SignatureUpdateProcessorFactoryTest, CopyFieldTest, NodeMutatorTest, ConfigSetsAPITest, AutoCommitTest, TestComplexPhraseQParserPlugin, HttpPartitionTest]
   [junit4] Completed [210/729 (1!)] on J2 in 34.79s, 1 test, 1 failure <<< FAILURES!

[...truncated 38292 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+181) - Build # 20296 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/20296/
Java: 64bit/jdk-9-ea+181 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC --illegal-access=deny

1 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
Expected 2 of 3 replicas to be active but only found 1; [core_node5:{"core":"c8n_1x3_lf_shard1_replica_n3","base_url":"http://127.0.0.1:40619/wgh/wa","node_name":"127.0.0.1:40619_wgh%2Fwa","state":"active","type":"NRT","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/27)={   "pullReplicas":"0",   "replicationFactor":"1",   "shards":{"shard1":{       "range":"80000000-7fffffff",       "state":"active",       "replicas":{         "core_node4":{           "core":"c8n_1x3_lf_shard1_replica_n1",           "base_url":"http://127.0.0.1:45277/wgh/wa",           "node_name":"127.0.0.1:45277_wgh%2Fwa",           "state":"down",           "type":"NRT"},         "core_node5":{           "core":"c8n_1x3_lf_shard1_replica_n3",           "base_url":"http://127.0.0.1:40619/wgh/wa",           "node_name":"127.0.0.1:40619_wgh%2Fwa",           "state":"active",           "type":"NRT",           "leader":"true"},         "core_node6":{           "state":"down",           "base_url":"http://127.0.0.1:39751/wgh/wa",           "core":"c8n_1x3_lf_shard1_replica_n2",           "node_name":"127.0.0.1:39751_wgh%2Fwa",           "type":"NRT"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"1",   "autoAddReplicas":"false",   "nrtReplicas":"3",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 1; [core_node5:{"core":"c8n_1x3_lf_shard1_replica_n3","base_url":"http://127.0.0.1:40619/wgh/wa","node_name":"127.0.0.1:40619_wgh%2Fwa","state":"active","type":"NRT","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/27)={
  "pullReplicas":"0",
  "replicationFactor":"1",
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{
        "core_node4":{
          "core":"c8n_1x3_lf_shard1_replica_n1",
          "base_url":"http://127.0.0.1:45277/wgh/wa",
          "node_name":"127.0.0.1:45277_wgh%2Fwa",
          "state":"down",
          "type":"NRT"},
        "core_node5":{
          "core":"c8n_1x3_lf_shard1_replica_n3",
          "base_url":"http://127.0.0.1:40619/wgh/wa",
          "node_name":"127.0.0.1:40619_wgh%2Fwa",
          "state":"active",
          "type":"NRT",
          "leader":"true"},
        "core_node6":{
          "state":"down",
          "base_url":"http://127.0.0.1:39751/wgh/wa",
          "core":"c8n_1x3_lf_shard1_replica_n2",
          "node_name":"127.0.0.1:39751_wgh%2Fwa",
          "type":"NRT"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"3",
  "tlogReplicas":"0"}
        at __randomizedtesting.SeedInfo.seed([508B4500DF551A5E:D8DF7ADA71A977A6]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:169)
        at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:56)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 1693 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/core/test/temp/junit4-J2-20170811_043122_0314881618963532299954.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 5 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/core/test/temp/junit4-J1-20170811_043122_0317771574985661907242.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/core/test/temp/junit4-J0-20170811_043122_0314280498430728886280.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 288 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/test-framework/test/temp/junit4-J1-20170811_043838_1309823645327314124372.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/test-framework/test/temp/junit4-J2-20170811_043838_1331131146662983713281.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/test-framework/test/temp/junit4-J0-20170811_043838_12617011624186300558321.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 1033 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/common/test/temp/junit4-J0-20170811_044037_8883569628955816543286.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/common/test/temp/junit4-J2-20170811_044037_8895686141794992784814.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/common/test/temp/junit4-J1-20170811_044037_8897271655267255226309.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 216 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/icu/test/temp/junit4-J1-20170811_044221_71515594843629356899768.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/icu/test/temp/junit4-J2-20170811_044221_71812376541932058689942.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 5 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/icu/test/temp/junit4-J0-20170811_044221_71413807589961295532506.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 231 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/kuromoji/test/temp/junit4-J0-20170811_044233_4458003487073574019008.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/kuromoji/test/temp/junit4-J2-20170811_044233_44517944952586545779349.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 9 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/kuromoji/test/temp/junit4-J1-20170811_044233_44517188423833053088299.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 146 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/morfologik/test/temp/junit4-J1-20170811_044255_8912280809742961242731.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/morfologik/test/temp/junit4-J0-20170811_044255_89110340739032050000815.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/morfologik/test/temp/junit4-J2-20170811_044255_89112192198928196248471.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 161 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/phonetic/test/temp/junit4-J2-20170811_044300_36417907629727524134654.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/phonetic/test/temp/junit4-J1-20170811_044300_3644460779654178000574.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/phonetic/test/temp/junit4-J0-20170811_044300_36413112729533059309837.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 145 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/smartcn/test/temp/junit4-J1-20170811_044311_0549693033979407137076.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/smartcn/test/temp/junit4-J0-20170811_044311_05412592308652796177901.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 149 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/stempel/test/temp/junit4-J2-20170811_044320_98418232858843152813102.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/stempel/test/temp/junit4-J1-20170811_044320_98513388593789017704623.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/stempel/test/temp/junit4-J0-20170811_044320_98415686854238291882639.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 161 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/uima/test/temp/junit4-J0-20170811_044324_4432221222995526681922.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 140 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/backward-codecs/test/temp/junit4-J1-20170811_044333_17011838707414122211032.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 5 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/backward-codecs/test/temp/junit4-J2-20170811_044333_17016410902805473785355.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 25 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/backward-codecs/test/temp/junit4-J0-20170811_044333_1703423961710890667609.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 1209 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/benchmark/test/temp/junit4-J1-20170811_044345_47416384472278456426957.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/benchmark/test/temp/junit4-J2-20170811_044345_47410650584642553589382.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/benchmark/test/temp/junit4-J0-20170811_044345_47416736320156576081597.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 217 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/classification/test/temp/junit4-J1-20170811_044356_4024467380954021473504.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/classification/test/temp/junit4-J2-20170811_044356_4023051253403468901214.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 6 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/classification/test/temp/junit4-J0-20170811_044356_4024619806260583000233.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 241 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/codecs/test/temp/junit4-J0-20170811_044411_1347274758679958863885.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 9 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/codecs/test/temp/junit4-J1-20170811_044411_1352936230517474328133.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 7 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/codecs/test/temp/junit4-J2-20170811_044411_13610124309545494307573.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 223 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/demo/test/temp/junit4-J1-20170811_044635_76717344575891668115524.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 6 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/demo/test/temp/junit4-J0-20170811_044635_76717240897654078983097.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/demo/test/temp/junit4-J2-20170811_044635_7678643522828478664989.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 162 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/expressions/test/temp/junit4-J0-20170811_044638_8807904857617046703948.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/expressions/test/temp/junit4-J1-20170811_044638_8807972868743777432410.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/expressions/test/temp/junit4-J2-20170811_044638_8806392635647131555539.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 207 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/facet/test/temp/junit4-J2-20170811_044645_3677763557898432519643.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/facet/test/temp/junit4-J1-20170811_044645_367781595124133018896.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/facet/test/temp/junit4-J0-20170811_044645_3624593621788962315456.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 171 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/grouping/test/temp/junit4-J2-20170811_044722_8615286568724146665709.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/grouping/test/temp/junit4-J1-20170811_044722_86117011566576387146535.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/grouping/test/temp/junit4-J0-20170811_044722_8615252779527918871236.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 232 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/highlighter/test/temp/junit4-J1-20170811_044739_11712820073178022902081.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/highlighter/test/temp/junit4-J2-20170811_044739_11812041974478858911125.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 9 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/highlighter/test/temp/junit4-J0-20170811_044739_1173506451740877723161.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 152 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/join/test/temp/junit4-J2-20170811_044754_7364179529306130620904.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 6 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/join/test/temp/junit4-J0-20170811_044754_73611927780769263599143.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/join/test/temp/junit4-J1-20170811_044754_73617892005436268639323.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 141 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/memory/test/temp/junit4-J0-20170811_044827_3983094274733442271738.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/memory/test/temp/junit4-J1-20170811_044827_3985752077277708073063.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 172 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/misc/test/temp/junit4-J2-20170811_044838_8911221971711664277669.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/misc/test/temp/junit4-J1-20170811_044838_8912157672572905007997.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 6 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/misc/test/temp/junit4-J0-20170811_044838_8915525328953910262435.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 304 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/queries/test/temp/junit4-J1-20170811_044858_316980175828313723197.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 6 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/queries/test/temp/junit4-J2-20170811_044858_3167015482589052346333.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/queries/test/temp/junit4-J0-20170811_044858_31611239106792051628329.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 216 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/queryparser/test/temp/junit4-J2-20170811_044907_02915652082654446467340.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 6 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/queryparser/test/temp/junit4-J0-20170811_044907_0297511766720740859764.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/queryparser/test/temp/junit4-J1-20170811_044907_0291856767221537456814.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 199 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/replicator/test/temp/junit4-J1-20170811_044915_23913823015161465830003.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/replicator/test/temp/junit4-J0-20170811_044915_2399611143323549423861.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/replicator/test/temp/junit4-J2-20170811_044915_2397041553572471192892.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 179 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/sandbox/test/temp/junit4-J1-20170811_044927_77411135847903607862182.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 7 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/sandbox/test/temp/junit4-J0-20170811_044927_77417298685982565565617.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 21 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/sandbox/test/temp/junit4-J2-20170811_044927_7817577623197214042804.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 210 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/spatial-extras/test/temp/junit4-J0-20170811_045013_0432875722843901901522.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/spatial-extras/test/temp/junit4-J1-20170811_045013_0439816901326931169829.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/spatial-extras/test/temp/junit4-J2-20170811_045013_0431102670701908974445.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 140 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/spatial/test/temp/junit4-J0-20170811_045027_0402828242901813366929.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 163 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/spatial3d/test/temp/junit4-J2-20170811_045028_9507536911093318295727.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/spatial3d/test/temp/junit4-J0-20170811_045028_95012228090249146651186.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 5 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/spatial3d/test/temp/junit4-J1-20170811_045028_95017360191053404236266.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 227 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/suggest/test/temp/junit4-J2-20170811_045127_0689086168695089871062.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/suggest/test/temp/junit4-J0-20170811_045127_06216355144612391892870.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 18 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/suggest/test/temp/junit4-J1-20170811_045127_0626160662281989794353.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 1585 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest
   [junit4]   2> 122335 INFO  (SUITE-LeaderFailoverAfterPartitionTest-seed#[508B4500DF551A5E]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/init-core-data-001
   [junit4]   2> 122335 WARN  (SUITE-LeaderFailoverAfterPartitionTest-seed#[508B4500DF551A5E]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 122335 INFO  (SUITE-LeaderFailoverAfterPartitionTest-seed#[508B4500DF551A5E]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 122335 INFO  (SUITE-LeaderFailoverAfterPartitionTest-seed#[508B4500DF551A5E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl="https://issues.apache.org/jira/browse/SOLR-5776")
   [junit4]   2> 122336 INFO  (SUITE-LeaderFailoverAfterPartitionTest-seed#[508B4500DF551A5E]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /wgh/wa
   [junit4]   2> 122336 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 122338 INFO  (Thread-356) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 122338 INFO  (Thread-356) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 122348 ERROR (Thread-356) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 122438 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.ZkTestServer start zk server on port:42407
   [junit4]   2> 122443 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 122444 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 122444 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 122445 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 122445 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 122445 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 122445 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 122446 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 122446 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 122446 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 122447 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 122447 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 122510 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 122510 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@68d4fa73{/wgh/wa,null,AVAILABLE}
   [junit4]   2> 122511 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@eb01b1f{HTTP/1.1,[http/1.1]}{127.0.0.1:38735}
   [junit4]   2> 122512 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.e.j.s.Server Started @124092ms
   [junit4]   2> 122512 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/wgh/wa, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/tempDir-001/control/data, hostPort=45277, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/control-001/cores}
   [junit4]   2> 122512 ERROR (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 122512 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 122512 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 122512 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 122512 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-11T04:54:38.224505Z
   [junit4]   2> 122514 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 122514 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/control-001/solr.xml
   [junit4]   2> 122520 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1e33f466, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 122522 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42407/solr
   [junit4]   2> 122551 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 122551 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45277_wgh%2Fwa
   [junit4]   2> 122551 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.Overseer Overseer (id=98463074094284804-127.0.0.1:45277_wgh%2Fwa-n_0000000000) starting
   [junit4]   2> 122553 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45277_wgh%2Fwa
   [junit4]   2> 122554 INFO  (OverseerStateUpdate-98463074094284804-127.0.0.1:45277_wgh%2Fwa-n_0000000000) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 122582 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e33f466
   [junit4]   2> 122586 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e33f466
   [junit4]   2> 122586 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e33f466
   [junit4]   2> 122588 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/control-001/cores
   [junit4]   2> 122616 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 122617 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42407/solr ready
   [junit4]   2> 122618 INFO  (SocketProxy-Acceptor-45277) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=47124,localport=45277], receiveBufferSize:531000
   [junit4]   2> 122624 INFO  (SocketProxy-Acceptor-45277) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=38735,localport=50532], receiveBufferSize=530904
   [junit4]   2> 122631 INFO  (qtp161806313-1313) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:45277_wgh%252Fwa&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 122644 INFO  (OverseerThreadFactory-396-thread-1) [    ] o.a.s.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 122773 INFO  (SocketProxy-Acceptor-45277) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=47128,localport=45277], receiveBufferSize:531000
   [junit4]   2> 122773 INFO  (SocketProxy-Acceptor-45277) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=38735,localport=50536], receiveBufferSize=530904
   [junit4]   2> 122774 INFO  (qtp161806313-1312) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 122774 INFO  (qtp161806313-1312) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 122877 INFO  (zkCallback-197-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 123786 INFO  (qtp161806313-1312) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 123798 INFO  (qtp161806313-1312) [    ] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 123886 INFO  (qtp161806313-1312) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 123903 INFO  (qtp161806313-1312) [    ] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 123903 INFO  (qtp161806313-1312) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e33f466
   [junit4]   2> 123904 INFO  (qtp161806313-1312) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 123904 INFO  (qtp161806313-1312) [    ] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 123907 INFO  (qtp161806313-1312) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=77.439453125, floorSegmentMB=0.44140625, forceMergeDeletesPctAllowed=15.268244141110046, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 123910 WARN  (qtp161806313-1312) [    ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 123950 INFO  (qtp161806313-1312) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 123950 INFO  (qtp161806313-1312) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 123952 INFO  (qtp161806313-1312) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 123952 INFO  (qtp161806313-1312) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 123956 INFO  (qtp161806313-1312) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=43.1962890625, floorSegmentMB=0.2646484375, forceMergeDeletesPctAllowed=29.346857319377307, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 123957 INFO  (qtp161806313-1312) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@ce00dfb[control_collection_shard1_replica_n1] main]
   [junit4]   2> 123958 INFO  (qtp161806313-1312) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 123958 INFO  (qtp161806313-1312) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 123959 INFO  (qtp161806313-1312) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 123959 INFO  (qtp161806313-1312) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575409187208298496
   [junit4]   2> 123964 INFO  (qtp161806313-1312) [    ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 123964 INFO  (qtp161806313-1312) [    ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 123964 INFO  (qtp161806313-1312) [    ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45277/wgh/wa/control_collection_shard1_replica_n1/
   [junit4]   2> 123964 INFO  (qtp161806313-1312) [    ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 123964 INFO  (qtp161806313-1312) [    ] o.a.s.c.SyncStrategy http://127.0.0.1:45277/wgh/wa/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 123964 INFO  (qtp161806313-1312) [    ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 123966 INFO  (qtp161806313-1312) [    ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45277/wgh/wa/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 123968 INFO  (searcherExecutor-399-thread-1) [    ] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@ce00dfb[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 124068 INFO  (zkCallback-197-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 124117 INFO  (qtp161806313-1312) [    ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 124118 INFO  (qtp161806313-1312) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1344
   [junit4]   2> 124120 INFO  (qtp161806313-1313) [    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 124220 INFO  (zkCallback-197-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 124645 INFO  (OverseerCollectionConfigSetProcessor-98463074094284804-127.0.0.1:45277_wgh%2Fwa-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 125120 INFO  (qtp161806313-1313) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:45277_wgh%252Fwa&wt=javabin&version=2} status=0 QTime=2489
   [junit4]   2> 125125 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 125125 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42407/solr ready
   [junit4]   2> 125125 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 125126 INFO  (SocketProxy-Acceptor-45277) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=47150,localport=45277], receiveBufferSize:531000
   [junit4]   2> 125126 INFO  (SocketProxy-Acceptor-45277) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=38735,localport=50558], receiveBufferSize=530904
   [junit4]   2> 125127 INFO  (qtp161806313-1313) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 125128 INFO  (OverseerThreadFactory-396-thread-2) [    ] o.a.s.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 125128 WARN  (OverseerThreadFactory-396-thread-2) [    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 125333 INFO  (qtp161806313-1313) [    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 125333 INFO  (qtp161806313-1313) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=206
   [junit4]   2> 125401 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/shard-1-001 of type NRT
   [junit4]   2> 125402 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 125403 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1ae2c58f{/wgh/wa,null,AVAILABLE}
   [junit4]   2> 125403 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@409725b3{HTTP/1.1,[http/1.1]}{127.0.0.1:39197}
   [junit4]   2> 125403 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.e.j.s.Server Started @126984ms
   [junit4]   2> 125403 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/wgh/wa, solrconfig=solrconfig.xml, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/tempDir-001/jetty1, hostPort=39751, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/shard-1-001/cores}
   [junit4]   2> 125403 ERROR (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 125403 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 125403 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 125403 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 125403 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-11T04:54:41.115762Z
   [junit4]   2> 125405 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 125405 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/shard-1-001/solr.xml
   [junit4]   2> 125408 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1e33f466, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 125410 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42407/solr
   [junit4]   2> 125415 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 125416 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 125417 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39751_wgh%2Fwa
   [junit4]   2> 125417 INFO  (zkCallback-197-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 125417 INFO  (zkCallback-204-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 125418 INFO  (zkCallback-209-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 125468 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e33f466
   [junit4]   2> 125472 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e33f466
   [junit4]   2> 125472 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e33f466
   [junit4]   2> 125473 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/shard-1-001/cores
   [junit4]   2> 125484 INFO  (qtp161806313-1315) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:39751_wgh%252Fwa&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 125485 INFO  (OverseerCollectionConfigSetProcessor-98463074094284804-127.0.0.1:45277_wgh%2Fwa-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 125485 INFO  (OverseerThreadFactory-396-thread-3) [    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:39751_wgh%2Fwa for creating new replica
   [junit4]   2> 125486 INFO  (SocketProxy-Acceptor-39751) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=41336,localport=39751], receiveBufferSize:531000
   [junit4]   2> 125488 INFO  (SocketProxy-Acceptor-39751) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39197,localport=43384], receiveBufferSize=530904
   [junit4]   2> 125489 INFO  (qtp1217495132-1361) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 125489 INFO  (qtp1217495132-1361) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 126498 INFO  (qtp1217495132-1361) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 126507 INFO  (qtp1217495132-1361) [    ] o.a.s.s.IndexSchema [collection1_shard2_replica_n41] Schema name=test
   [junit4]   2> 126562 INFO  (qtp1217495132-1361) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 126589 INFO  (qtp1217495132-1361) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n41' using configuration from collection collection1, trusted=true
   [junit4]   2> 126589 INFO  (qtp1217495132-1361) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n41' (registry 'solr.core.collection1.shard2.replica_n41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e33f466
   [junit4]   2> 126589 INFO  (qtp1217495132-1361) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 126589 INFO  (qtp1217495132-1361) [    ] o.a.s.c.SolrCore [[collection1_shard2_replica_n41] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/shard-1-001/cores/collection1_shard2_replica_n41], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/shard-1-001/cores/collection1_shard2_replica_n41/data/]
   [junit4]   2> 126591 INFO  (qtp1217495132-1361) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=77.439453125, floorSegmentMB=0.44140625, forceMergeDeletesPctAllowed=15.268244141110046, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 126592 WARN  (qtp1217495132-1361) [    ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 126615 INFO  (qtp1217495132-1361) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 126615 INFO  (qtp1217495132-1361) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 126616 INFO  (qtp1217495132-1361) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 126616 INFO  (qtp1217495132-1361) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 126616 INFO  (qtp1217495132-1361) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=43.1962890625, floorSegmentMB=0.2646484375, forceMergeDeletesPctAllowed=29.346857319377307, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 126617 INFO  (qtp1217495132-1361) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1aadc98a[collection1_shard2_replica_n41] main]
   [junit4]   2> 126617 INFO  (qtp1217495132-1361) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 126618 INFO  (qtp1217495132-1361) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 126618 INFO  (qtp1217495132-1361) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 126619 INFO  (searcherExecutor-410-thread-1) [    ] o.a.s.c.SolrCore [collection1_shard2_replica_n41] Registered new searcher Searcher@1aadc98a[collection1_shard2_replica_n41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 126619 INFO  (qtp1217495132-1361) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575409189997510656
   [junit4]   2> 126622 INFO  (qtp1217495132-1361) [    ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 126622 INFO  (qtp1217495132-1361) [    ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 126622 INFO  (qtp1217495132-1361) [    ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39751/wgh/wa/collection1_shard2_replica_n41/
   [junit4]   2> 126622 INFO  (qtp1217495132-1361) [    ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 126622 INFO  (qtp1217495132-1361) [    ] o.a.s.c.SyncStrategy http://127.0.0.1:39751/wgh/wa/collection1_shard2_replica_n41/ has no replicas
   [junit4]   2> 126622 INFO  (qtp1217495132-1361) [    ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 126623 INFO  (qtp1217495132-1361) [    ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39751/wgh/wa/collection1_shard2_replica_n41/ shard2
   [junit4]   2> 126773 INFO  (qtp1217495132-1361) [    ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 126776 INFO  (qtp1217495132-1361) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1287
   [junit4]   2> 126778 INFO  (qtp161806313-1315) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:39751_wgh%252Fwa&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1294
   [junit4]   2> 126878 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/shard-2-001 of type NRT
   [junit4]   2> 126878 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 126882 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@69abda6e{/wgh/wa,null,AVAILABLE}
   [junit4]   2> 126882 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@49b42cf5{HTTP/1.1,[http/1.1]}{127.0.0.1:41633}
   [junit4]   2> 126882 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.e.j.s.Server Started @128463ms
   [junit4]   2> 126882 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/wgh/wa, solrconfig=solrconfig.xml, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/tempDir-001/jetty2, hostPort=32837, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/shard-2-001/cores}
   [junit4]   2> 126883 ERROR (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 126883 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 126883 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 126883 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 126883 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-11T04:54:42.595575Z
   [junit4]   2> 126885 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 126886 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/shard-2-001/solr.xml
   [junit4]   2> 126889 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1e33f466, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 126893 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42407/solr
   [junit4]   2> 126899 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 126900 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 126905 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32837_wgh%2Fwa
   [junit4]   2> 126905 INFO  (zkCallback-209-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 126905 INFO  (zkCallback-204-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 126906 INFO  (zkCallback-197-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 126906 INFO  (zkCallback-215-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 126933 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e33f466
   [junit4]   2> 126936 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e33f466
   [junit4]   2> 126936 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e33f466
   [junit4]   2> 126937 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[508B4500DF551A5E]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_508B4500DF551A5E-001/shard-2-001/cores
   [junit4]   2> 126953 INFO  (qtp161806313-1312) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:32837_wgh%252Fwa&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 126954 INFO  (OverseerCollectionConfigSetProcessor-98463074094284804-127.0.0.1:45277_wgh%2Fwa-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 126954 INFO  (OverseerThreadFactory-396-thread-4) [    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:32837_wgh%2Fwa for creating new replica
   [junit4]   2> 126955 INFO  (SocketProxy-Acceptor-32837) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=58696,localport=32837], receiveBufferSize:531000
   [junit4]   2> 126956 INFO  (SocketProxy-Acceptor-32837) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=41633,localport=45210], receiveBufferSize=530904
   [junit4]   2> 126957 INFO  (qtp1551825623-1394) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 126958 INFO  (qtp1551825623-1394) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 127967 INFO  (qtp1551825623-1394) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 127994 INFO  (qtp1551825623-1394) [    ] o.a.s.s.IndexSchema [collection1_shard1_replica_n43] Schema name=test
   [junit4]   2> 128050 INFO  (qtp1551825623-1394) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 128058 INFO  (qtp1551825623-1394) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n43' using configuration from collection collection1, trusted=true
   [junit4]   2> 128059 INFO  (qtp1551825623-1394) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n43' (registry 'solr.core.collection1.shard1.replica_n43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e33f466
   [junit4]   2> 128059 INFO  (qtp1551825623-1394) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 128059 INFO  (qtp1551825623-1394) [    ] o.a.s.c.SolrCore [[collection1_shard1_replica_n43] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-

[...truncated too long message...]

ns/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-analytics/test/temp/junit4-J1-20170811_053702_1726865399721012325685.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-analytics/test/temp/junit4-J2-20170811_053702_1723719643961729591940.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 499 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-clustering/test/temp/junit4-J2-20170811_053733_6873323204671020624185.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-clustering/test/temp/junit4-J1-20170811_053733_68735478583628029491.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-clustering/test/temp/junit4-J0-20170811_053733_68711788648011635397422.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 1131 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-dataimporthandler-extras/test/temp/junit4-J1-20170811_053751_115395295987888210546.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-dataimporthandler-extras/test/temp/junit4-J0-20170811_053751_1153511154708166668181.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 596 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-dataimporthandler/test/temp/junit4-J2-20170811_053757_51410937529777168489407.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-dataimporthandler/test/temp/junit4-J1-20170811_053757_51411088854155870898021.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-dataimporthandler/test/temp/junit4-J0-20170811_053757_51416415621765093628113.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 503 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-cell/test/temp/junit4-J0-20170811_053820_95218306890278481688695.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-cell/test/temp/junit4-J2-20170811_053820_95215940253252273998801.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-cell/test/temp/junit4-J1-20170811_053820_952107453121871034905.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 504 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-langid/test/temp/junit4-J1-20170811_053831_2852512671586076060594.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-langid/test/temp/junit4-J0-20170811_053831_2855189095308311117857.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 592 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-ltr/test/temp/junit4-J1-20170811_053837_96312625263369636342739.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-ltr/test/temp/junit4-J0-20170811_053837_9635603658018409198234.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-ltr/test/temp/junit4-J2-20170811_053837_96310214279221613888611.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----

[...truncated 538 lines...]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-uima/test/temp/junit4-J1-20170811_053911_9645611132921435750476.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-uima/test/temp/junit4-J0-20170811_053911_9645097154951619555359.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 494 lines...]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-velocity/test/temp/junit4-J0-20170811_053922_789327467449298242701.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 28962 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_144) - Build # 20297 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/20297/
Java: 64bit/jdk1.8.0_144 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.update.HardAutoCommitTest.testCommitWithin

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
        at __randomizedtesting.SeedInfo.seed([3778A826EFABF4EE:8DAAC75E6C851AFB]:0)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:886)
        at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:100)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
        xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>

        request was:q=id:529&qt=&start=0&rows=20&version=2.2
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:879)
        ... 40 more




Build Log:
[...truncated 11982 lines...]
   [junit4] Suite: org.apache.solr.update.HardAutoCommitTest
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.update.HardAutoCommitTest_3778A826EFABF4EE-001/init-core-data-001
   [junit4]   2> 1384125 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1384126 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1384126 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1384126 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 1384127 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1384152 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1384184 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 1384250 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1384283 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3fb73da2
   [junit4]   2> 1384286 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3fb73da2
   [junit4]   2> 1384286 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3fb73da2
   [junit4]   2> 1384288 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1384318 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1384337 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1384533 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1384564 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 1384565 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3fb73da2
   [junit4]   2> 1384565 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1384565 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.update.HardAutoCommitTest_3778A826EFABF4EE-001/init-core-data-001/]
   [junit4]   2> 1384567 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=10, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.0380859375, noCFSRatio=1.0]
   [junit4]   2> 1384615 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1384615 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1384616 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1384616 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1384616 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@3c1b70e
   [junit4]   2> 1384616 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6d1da5de[collection1] main]
   [junit4]   2> 1384617 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1384637 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1384712 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1384713 INFO  (searcherExecutor-3886-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1384713 INFO  (coreLoadExecutor-3885-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575425286735921152
   [junit4]   2> 1384713 INFO  (searcherExecutor-3886-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 1384713 INFO  (searcherExecutor-3886-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1384713 INFO  (searcherExecutor-3886-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1384713 INFO  (searcherExecutor-3886-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1384713 INFO  (searcherExecutor-3886-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 1384713 INFO  (searcherExecutor-3886-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 1384713 INFO  (searcherExecutor-3886-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 1384713 INFO  (searcherExecutor-3886-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 1384715 INFO  (searcherExecutor-3886-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6d1da5de[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1384716 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 1384719 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCommitWithin
   [junit4]   2> 1384719 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 1384725 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@58bb60e9[collection1] realtime]
   [junit4]   2> 1384726 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 6
   [junit4]   2> 1384726 ERROR (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    ] o.a.s.c.CorePropertiesLocator Couldn't load core descriptor from /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/core.properties:java.nio.file.NoSuchFileException: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/core.properties
   [junit4]   2> 1384726 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1384753 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1384779 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1384963 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1384971 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1
   [junit4]   2> 1384989 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3fb73da2
   [junit4]   2> 1384989 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.update.HardAutoCommitTest_3778A826EFABF4EE-001/init-core-data-001/]
   [junit4]   2> 1385048 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1385048 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1385049 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7fe2e0e2[collection1] main]
   [junit4]   2> 1385050 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1385050 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1385053 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1385053 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575425287092436992
   [junit4]   2> 1385064 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=45, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.36207947396391926]
   [junit4]   2> 1385064 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1385064 INFO  (searcherExecutor-3891-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1385065 INFO  (searcherExecutor-3891-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 1385065 INFO  (searcherExecutor-3891-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1385065 INFO  (searcherExecutor-3891-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1385065 INFO  (searcherExecutor-3891-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1385065 INFO  (searcherExecutor-3891-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 1385065 INFO  (searcherExecutor-3891-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 1385065 INFO  (searcherExecutor-3891-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 1385065 INFO  (searcherExecutor-3891-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 1385067 INFO  (searcherExecutor-3891-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7fe2e0e2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1385068 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6cec39ea[collection1] main]
   [junit4]   2> 1385068 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@300a8d89
   [junit4]   2> 1385068 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=805997961
   [junit4]   2> 1385076 INFO  (searcherExecutor-3891-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6cec39ea[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1385077 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[529 (1575425287114457088)]} 0 2
   [junit4]   2> 1385077 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1385079 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[530 (1575425287119699968)]} 0 5
   [junit4]   2> 1385080 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:530&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1385085 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[529 (-1575425287125991424)]} 0 11
   [junit4]   2> 1385085 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1385086 ERROR (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2>
   [junit4]   2> request was:q=id:529&qt=&start=0&rows=20&version=2.2
   [junit4]   2> 1385086 ERROR (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: q=id:529&qt=&start=0&rows=20&version=2.2:java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2>
   [junit4]   2> request was:q=id:529&qt=&start=0&rows=20&version=2.2
   [junit4]   2> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:879)
   [junit4]   2> at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:100)
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
   [junit4]   2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>
   [junit4]   2> 1385087 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[3778A826EFABF4EE]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testCommitWithin
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HardAutoCommitTest -Dtests.method=testCommitWithin -Dtests.seed=3778A826EFABF4EE -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar-KW -Dtests.timezone=America/St_Thomas -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.38s J0 | HardAutoCommitTest.testCommitWithin <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: Exception during query
   [junit4]    > at __randomizedtesting.SeedInfo.seed([3778A826EFABF4EE:8DAAC75E6C851AFB]:0)
   [junit4]    > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:886)
   [junit4]    > at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:100)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]    > xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]    > <response>
   [junit4]    > <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]    > </response>
   [junit4]    > request was:q=id:529&qt=&start=0&rows=20&version=2.2
   [junit4]    > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:879)
   [junit4]    > ... 40 more
   [junit4]   2> 1385092 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 1385092 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    x:collection1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2053301270
   [junit4]   2> 1385092 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1385094 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1385095 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1385095 INFO  (SUITE-HardAutoCommitTest-seed#[3778A826EFABF4EE]-worker) [    x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1385103 INFO  (coreCloseExecutor-3895-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1c1d3f9a
   [junit4]   2> 1385103 INFO  (coreCloseExecutor-3895-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter close.
   [junit4]   2> 1385103 INFO  (coreCloseExecutor-3895-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@749f1682 commitCommandVersion:0
   [junit4]   2> 1385120 INFO  (coreCloseExecutor-3895-thread-1-processing-x:collection1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=471678874
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.update.HardAutoCommitTest_3778A826EFABF4EE-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), sim=RandomSimilarity(queryNorm=true): {}, locale=ar-KW, timezone=America/St_Thomas
   [junit4]   2> NOTE: Linux 4.10.0-27-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=8,threads=1,free=172580392,total=529530880
   [junit4]   2> NOTE: All tests run in this JVM: [TestCollectionsAPIViaSolrCloudCluster, TestRangeQuery, HighlighterConfigTest, TestCopyFieldCollectionResource, TestSystemCollAutoCreate, BlockCacheTest, TestRandomRequestDistribution, TestMinMaxOnMultiValuedField, TestRemoteStreaming, TestInitParams, TestTlogReplica, OverseerModifyCollectionTest, IndexBasedSpellCheckerTest, SolrPluginUtilsTest, TestNestedDocsSort, CoreAdminCreateDiscoverTest, SaslZkACLProviderTest, SolrIndexSplitterTest, TestConfigSetsAPIZkFailure, TestLuceneMatchVersion, DistributedQueryElevationComponentTest, SolrIndexConfigTest, TestXmlQParserPlugin, TestManagedResourceStorage, SolrCoreTest, BasicDistributedZkTest, TestBadConfig, TestConfigOverlay, TestCoreAdminApis, AtomicUpdateProcessorFactoryTest, ChaosMonkeyNothingIsSafeTest, PeerSyncTest, TestImpersonationWithHadoopAuth, TestSubQueryTransformerCrossCore, ForceLeaderTest, TestLegacyNumericRangeQueryBuilder, UUIDFieldTest, TestManagedSchemaAPI, DeleteInactiveReplicaTest, TestGraphTermsQParserPlugin, BasicZkTest, TestNumericRangeQuery32, ZkStateReaderTest, BigEndianAscendingWordSerializerTest, CursorPagingTest, TestSegmentSorting, CdcrReplicationHandlerTest, TestOmitPositions, RestartWhileUpdatingTest, TestPerFieldSimilarityWithDefaultOverride, DistributedQueueTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestSolrDeletionPolicy1, TestRTimerTree, TestFileDictionaryLookup, TestExtendedDismaxParser, TestSolr4Spatial, TestStressCloudBlindAtomicUpdates, TestLegacyNumericUtils, TestFieldCache, TestIBSimilarityFactory, DefaultValueUpdateProcessorTest, TestCollapseQParserPlugin, ChaosMonkeyNothingIsSafeWithPullReplicasTest, SecurityConfHandlerTest, TestWordDelimiterFilterFactory, TestRandomCollapseQParserPlugin, OpenExchangeRatesOrgProviderTest, TestRandomDVFaceting, DistribCursorPagingTest, TestBinaryField, DocExpirationUpdateProcessorFactoryTest, TestSolrQueryParser, ExternalFileFieldSortTest, CdcrUpdateLogTest, TestSchemaManager, TestReqParamsAPI, RequestLoggingTest, TestSolrCloudWithDelegationTokens, InfoHandlerTest, RAMDirectoryFactoryTest, TestLeaderInitiatedRecoveryThread, TestFunctionQuery, SuggestComponentTest, MissingSegmentRecoveryTest, SpellCheckCollatorTest, EchoParamsTest, TestCSVResponseWriter, TestMacros, SolrIndexMetricsTest, DistanceUnitsTest, PreAnalyzedUpdateProcessorTest, DocumentAnalysisRequestHandlerTest, TestPayloadCheckQParserPlugin, DistributedFacetPivotSmallAdvancedTest, HdfsRecoveryZkTest, TestCrossCoreJoin, HdfsBasicDistributedZkTest, HdfsWriteToMultipleCollectionsTest, DeleteReplicaTest, SyncSliceTest, TestClassicSimilarityFactory, SolrCoreMetricManagerTest, TestApiFramework, SpellingQueryConverterTest, WordBreakSolrSpellCheckerTest, PreAnalyzedFieldManagedSchemaCloudTest, TestConfigSets, TestFaceting, DistributedQueryComponentCustomSortTest, TlogReplayBufferedWhileIndexingTest, TestComponentsName, TestHighlightDedupGrouping, HdfsCollectionsAPIDistributedZkTest, OverseerTaskQueueTest, TestLRUStatsCache, TestIntervalFaceting, TestMergePolicyConfig, AutoCommitTest, ZkControllerTest, HardAutoCommitTest]
   [junit4] Completed [375/729 (1!)] on J0 in 1.01s, 1 test, 1 error <<< FAILURES!

[...truncated 44412 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_144) - Build # 20298 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/20298/
Java: 64bit/jdk1.8.0_144 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
Doc with id=1 not found in http://127.0.0.1:37079/forceleader_test_collection due to: Path not found: /id; rsp={doc=null}

Stack Trace:
java.lang.AssertionError: Doc with id=1 not found in http://127.0.0.1:37079/forceleader_test_collection due to: Path not found: /id; rsp={doc=null}
        at __randomizedtesting.SeedInfo.seed([24D1FBB0306F6AC6:C246CF7009ED93A7]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:603)
        at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:556)
        at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:142)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete

Error Message:
Error from server at http://127.0.0.1:35085/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 </title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason: <pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/> </body> </html>

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:35085/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
</body>
</html>

        at __randomizedtesting.SeedInfo.seed([24D1FBB0306F6AC6:872B5515B7878063]:0)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
        at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
        at org.apache.solr.cloud.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete(TestCollectionsAPIViaSolrCloudCluster.java:167)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:35085/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
</body>
</html>

        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:591)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:516)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        ... 1 more




Build Log:
[...truncated 11081 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestCollectionsAPIViaSolrCloudCluster
   [junit4]   2> 300611 INFO  (SUITE-TestCollectionsAPIViaSolrCloudCluster-seed#[24D1FBB0306F6AC6]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_24D1FBB0306F6AC6-001/init-core-data-001
   [junit4]   2> 300612 INFO  (SUITE-TestCollectionsAPIViaSolrCloudCluster-seed#[24D1FBB0306F6AC6]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 300612 INFO  (SUITE-TestCollectionsAPIViaSolrCloudCluster-seed#[24D1FBB0306F6AC6]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 300614 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 5 servers in /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_24D1FBB0306F6AC6-001/tempDir-001
   [junit4]   2> 300614 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 300614 INFO  (Thread-606) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 300614 INFO  (Thread-606) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 300624 ERROR (Thread-606) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 300714 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.c.ZkTestServer start zk server on port:32799
   [junit4]   2> 300726 INFO  (jetty-launcher-437-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 300727 INFO  (jetty-launcher-437-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 300727 INFO  (jetty-launcher-437-thread-4) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 300728 INFO  (jetty-launcher-437-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1e80c11f{/solr,null,AVAILABLE}
   [junit4]   2> 300729 INFO  (jetty-launcher-437-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@62c7534e{HTTP/1.1,[http/1.1]}{127.0.0.1:34069}
   [junit4]   2> 300729 INFO  (jetty-launcher-437-thread-2) [    ] o.e.j.s.Server Started @302079ms
   [junit4]   2> 300729 INFO  (jetty-launcher-437-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34069}
   [junit4]   2> 300730 INFO  (jetty-launcher-437-thread-3) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 300730 ERROR (jetty-launcher-437-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 300730 INFO  (jetty-launcher-437-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 300730 INFO  (jetty-launcher-437-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 300730 INFO  (jetty-launcher-437-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 300730 INFO  (jetty-launcher-437-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-11T12:56:12.718Z
   [junit4]   2> 300731 INFO  (jetty-launcher-437-thread-5) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 300733 INFO  (jetty-launcher-437-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@e01678f{/solr,null,AVAILABLE}
   [junit4]   2> 300741 INFO  (jetty-launcher-437-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@253921f9{HTTP/1.1,[http/1.1]}{127.0.0.1:35085}
   [junit4]   2> 300741 INFO  (jetty-launcher-437-thread-1) [    ] o.e.j.s.Server Started @302090ms
   [junit4]   2> 300741 INFO  (jetty-launcher-437-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35085}
   [junit4]   2> 300741 ERROR (jetty-launcher-437-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 300741 INFO  (jetty-launcher-437-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 300741 INFO  (jetty-launcher-437-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 300741 INFO  (jetty-launcher-437-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 300741 INFO  (jetty-launcher-437-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-11T12:56:12.729Z
   [junit4]   2> 300746 INFO  (jetty-launcher-437-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@a25186f{/solr,null,AVAILABLE}
   [junit4]   2> 300747 INFO  (jetty-launcher-437-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@121f4826{HTTP/1.1,[http/1.1]}{127.0.0.1:44885}
   [junit4]   2> 300747 INFO  (jetty-launcher-437-thread-4) [    ] o.e.j.s.Server Started @302096ms
   [junit4]   2> 300747 INFO  (jetty-launcher-437-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44885}
   [junit4]   2> 300747 ERROR (jetty-launcher-437-thread-4) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 300747 INFO  (jetty-launcher-437-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 300747 INFO  (jetty-launcher-437-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 300747 INFO  (jetty-launcher-437-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 300747 INFO  (jetty-launcher-437-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-11T12:56:12.735Z
   [junit4]   2> 300749 INFO  (jetty-launcher-437-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 300753 INFO  (jetty-launcher-437-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@63883fab{/solr,null,AVAILABLE}
   [junit4]   2> 300753 INFO  (jetty-launcher-437-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@6128006b{HTTP/1.1,[http/1.1]}{127.0.0.1:39201}
   [junit4]   2> 300753 INFO  (jetty-launcher-437-thread-3) [    ] o.e.j.s.Server Started @302102ms
   [junit4]   2> 300753 INFO  (jetty-launcher-437-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39201}
   [junit4]   2> 300753 ERROR (jetty-launcher-437-thread-3) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 300754 INFO  (jetty-launcher-437-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 300754 INFO  (jetty-launcher-437-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 300754 INFO  (jetty-launcher-437-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 300754 INFO  (jetty-launcher-437-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-11T12:56:12.742Z
   [junit4]   2> 300764 INFO  (jetty-launcher-437-thread-5) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7dc365a4{/solr,null,AVAILABLE}
   [junit4]   2> 300764 INFO  (jetty-launcher-437-thread-5) [    ] o.e.j.s.AbstractConnector Started ServerConnector@447a9b90{HTTP/1.1,[http/1.1]}{127.0.0.1:42317}
   [junit4]   2> 300764 INFO  (jetty-launcher-437-thread-5) [    ] o.e.j.s.Server Started @302113ms
   [junit4]   2> 300764 INFO  (jetty-launcher-437-thread-5) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42317}
   [junit4]   2> 300764 ERROR (jetty-launcher-437-thread-5) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 300764 INFO  (jetty-launcher-437-thread-5) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 300764 INFO  (jetty-launcher-437-thread-5) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 300764 INFO  (jetty-launcher-437-thread-5) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 300765 INFO  (jetty-launcher-437-thread-5) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-11T12:56:12.752Z
   [junit4]   2> 300767 INFO  (jetty-launcher-437-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 300768 INFO  (jetty-launcher-437-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 300768 INFO  (jetty-launcher-437-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 300768 INFO  (jetty-launcher-437-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32799/solr
   [junit4]   2> 300775 INFO  (jetty-launcher-437-thread-5) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 300785 INFO  (jetty-launcher-437-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32799/solr
   [junit4]   2> 300814 INFO  (jetty-launcher-437-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32799/solr
   [junit4]   2> 300820 INFO  (jetty-launcher-437-thread-5) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32799/solr
   [junit4]   2> 300821 INFO  (jetty-launcher-437-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32799/solr
   [junit4]   2> 300828 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15dd15dd03b0007, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 300908 INFO  (jetty-launcher-437-thread-5) [n:127.0.0.1:42317_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 300908 INFO  (jetty-launcher-437-thread-5) [n:127.0.0.1:42317_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42317_solr
   [junit4]   2> 300909 INFO  (jetty-launcher-437-thread-5) [n:127.0.0.1:42317_solr    ] o.a.s.c.Overseer Overseer (id=98464967727710223-127.0.0.1:42317_solr-n_0000000000) starting
   [junit4]   2> 300919 INFO  (jetty-launcher-437-thread-3) [n:127.0.0.1:39201_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 300919 INFO  (jetty-launcher-437-thread-5) [n:127.0.0.1:42317_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42317_solr
   [junit4]   2> 300921 INFO  (jetty-launcher-437-thread-3) [n:127.0.0.1:39201_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39201_solr
   [junit4]   2> 300924 INFO  (zkCallback-464-thread-1-processing-n:127.0.0.1:42317_solr) [n:127.0.0.1:42317_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 300932 INFO  (jetty-launcher-437-thread-1) [n:127.0.0.1:35085_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 300934 INFO  (jetty-launcher-437-thread-1) [n:127.0.0.1:35085_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 300936 INFO  (jetty-launcher-437-thread-1) [n:127.0.0.1:35085_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35085_solr
   [junit4]   2> 300948 INFO  (zkCallback-464-thread-1-processing-n:127.0.0.1:42317_solr) [n:127.0.0.1:42317_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
   [junit4]   2> 300964 INFO  (zkCallback-463-thread-2-processing-n:127.0.0.1:39201_solr) [n:127.0.0.1:39201_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 300967 INFO  (zkCallback-462-thread-1-processing-n:127.0.0.1:35085_solr) [n:127.0.0.1:35085_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 300976 INFO  (jetty-launcher-437-thread-4) [n:127.0.0.1:44885_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 300980 INFO  (jetty-launcher-437-thread-4) [n:127.0.0.1:44885_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 300986 INFO  (jetty-launcher-437-thread-4) [n:127.0.0.1:44885_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44885_solr
   [junit4]   2> 300986 INFO  (zkCallback-462-thread-1-processing-n:127.0.0.1:35085_solr) [n:127.0.0.1:35085_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 300986 INFO  (zkCallback-463-thread-1-processing-n:127.0.0.1:39201_solr) [n:127.0.0.1:39201_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 300986 INFO  (zkCallback-464-thread-2-processing-n:127.0.0.1:42317_solr) [n:127.0.0.1:42317_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 300987 INFO  (zkCallback-461-thread-1-processing-n:127.0.0.1:44885_solr) [n:127.0.0.1:44885_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 300991 INFO  (jetty-launcher-437-thread-5) [n:127.0.0.1:42317_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42317.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 300995 INFO  (jetty-launcher-437-thread-5) [n:127.0.0.1:42317_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42317.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 300995 INFO  (jetty-launcher-437-thread-5) [n:127.0.0.1:42317_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42317.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 300996 INFO  (jetty-launcher-437-thread-5) [n:127.0.0.1:42317_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_24D1FBB0306F6AC6-001/tempDir-001/node5/.
   [junit4]   2> 301031 INFO  (jetty-launcher-437-thread-3) [n:127.0.0.1:39201_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39201.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 301033 INFO  (jetty-launcher-437-thread-2) [n:127.0.0.1:34069_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 301033 INFO  (jetty-launcher-437-thread-3) [n:127.0.0.1:39201_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39201.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 301034 INFO  (jetty-launcher-437-thread-3) [n:127.0.0.1:39201_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39201.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 301034 INFO  (jetty-launcher-437-thread-2) [n:127.0.0.1:34069_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 301035 INFO  (jetty-launcher-437-thread-2) [n:127.0.0.1:34069_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34069_solr
   [junit4]   2> 301036 INFO  (zkCallback-463-thread-1-processing-n:127.0.0.1:39201_solr) [n:127.0.0.1:39201_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 301036 INFO  (zkCallback-462-thread-1-processing-n:127.0.0.1:35085_solr) [n:127.0.0.1:35085_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 301036 INFO  (zkCallback-464-thread-2-processing-n:127.0.0.1:42317_solr) [n:127.0.0.1:42317_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 301036 INFO  (zkCallback-461-thread-1-processing-n:127.0.0.1:44885_solr) [n:127.0.0.1:44885_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 301037 INFO  (jetty-launcher-437-thread-3) [n:127.0.0.1:39201_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_24D1FBB0306F6AC6-001/tempDir-001/node4/.
   [junit4]   2> 301038 INFO  (jetty-launcher-437-thread-1) [n:127.0.0.1:35085_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35085.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 301042 INFO  (jetty-launcher-437-thread-1) [n:127.0.0.1:35085_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35085.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 301042 INFO  (jetty-launcher-437-thread-1) [n:127.0.0.1:35085_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35085.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 301043 INFO  (jetty-launcher-437-thread-1) [n:127.0.0.1:35085_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_24D1FBB0306F6AC6-001/tempDir-001/node1/.
   [junit4]   2> 301049 INFO  (jetty-launcher-437-thread-4) [n:127.0.0.1:44885_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44885.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 301052 INFO  (jetty-launcher-437-thread-4) [n:127.0.0.1:44885_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44885.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 301052 INFO  (jetty-launcher-437-thread-4) [n:127.0.0.1:44885_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44885.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 301068 INFO  (zkCallback-460-thread-1-processing-n:127.0.0.1:34069_solr) [n:127.0.0.1:34069_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 301078 INFO  (jetty-launcher-437-thread-4) [n:127.0.0.1:44885_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_24D1FBB0306F6AC6-001/tempDir-001/node3/.
   [junit4]   2> 301178 INFO  (jetty-launcher-437-thread-2) [n:127.0.0.1:34069_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34069.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 301182 INFO  (jetty-launcher-437-thread-2) [n:127.0.0.1:34069_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34069.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 301182 INFO  (jetty-launcher-437-thread-2) [n:127.0.0.1:34069_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34069.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 301183 INFO  (jetty-launcher-437-thread-2) [n:127.0.0.1:34069_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_24D1FBB0306F6AC6-001/tempDir-001/node2/.
   [junit4]   2> 301252 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
   [junit4]   2> 301253 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:32799/solr ready
   [junit4]   2> 301257 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCollectionCreateSearchDelete
   [junit4]   2> 301267 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@447a9b90{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 301267 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=710217207
   [junit4]   2> 301267 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 301269 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 301271 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 301273 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 301276 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 301276 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.c.Overseer Overseer (id=98464967727710223-127.0.0.1:42317_solr-n_0000000000) closing
   [junit4]   2> 301276 INFO  (OverseerStateUpdate-98464967727710223-127.0.0.1:42317_solr-n_0000000000) [n:127.0.0.1:42317_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42317_solr
   [junit4]   2> 301278 INFO  (zkCallback-463-thread-1-processing-n:127.0.0.1:39201_solr) [n:127.0.0.1:39201_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39201_solr
   [junit4]   2> 301278 INFO  (zkCallback-462-thread-1-processing-n:127.0.0.1:35085_solr) [n:127.0.0.1:35085_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 301278 INFO  (zkCallback-460-thread-1-processing-n:127.0.0.1:34069_solr) [n:127.0.0.1:34069_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 301278 INFO  (zkCallback-463-thread-2-processing-n:127.0.0.1:39201_solr) [n:127.0.0.1:39201_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 301279 INFO  (zkCallback-473-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 301278 INFO  (zkCallback-461-thread-1-processing-n:127.0.0.1:44885_solr) [n:127.0.0.1:44885_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 301279 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7dc365a4{/solr,null,UNAVAILABLE}
   [junit4]   2> 301279 INFO  (zkCallback-463-thread-1-processing-n:127.0.0.1:39201_solr) [n:127.0.0.1:39201_solr    ] o.a.s.c.Overseer Overseer (id=98464967727710222-127.0.0.1:39201_solr-n_0000000001) starting
   [junit4]   2> 301281 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 301288 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@fc4b7f0{/solr,null,AVAILABLE}
   [junit4]   2> 301288 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@607e3601{HTTP/1.1,[http/1.1]}{127.0.0.1:37699}
   [junit4]   2> 301288 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.e.j.s.Server Started @302637ms
   [junit4]   2> 301288 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37699}
   [junit4]   2> 301289 ERROR (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 301290 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 301290 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 301290 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 301290 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-11T12:56:13.278Z
   [junit4]   2> 301294 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 301301 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32799/solr
   [junit4]   2> 301315 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [n:127.0.0.1:37699_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 301316 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [n:127.0.0.1:37699_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 301317 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [n:127.0.0.1:37699_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37699_solr
   [junit4]   2> 301318 INFO  (zkCallback-460-thread-1-processing-n:127.0.0.1:34069_solr) [n:127.0.0.1:34069_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 301318 INFO  (zkCallback-461-thread-1-processing-n:127.0.0.1:44885_solr) [n:127.0.0.1:44885_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 301319 INFO  (zkCallback-462-thread-1-processing-n:127.0.0.1:35085_solr) [n:127.0.0.1:35085_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 301319 INFO  (zkCallback-473-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 301319 INFO  (zkCallback-463-thread-1-processing-n:127.0.0.1:39201_solr) [n:127.0.0.1:39201_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 301324 INFO  (zkCallback-478-thread-1-processing-n:127.0.0.1:37699_solr) [n:127.0.0.1:37699_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 301403 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [n:127.0.0.1:37699_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37699.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 301407 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [n:127.0.0.1:37699_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37699.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 301408 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [n:127.0.0.1:37699_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37699.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 301409 INFO  (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[24D1FBB0306F6AC6]) [n:127.0.0.1:37699_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_24D1FBB0306F6AC6-001/tempDir-001/node6/.
   [junit4]   2> 301461 INFO  (qtp2024517883-2906) [n:127.0.0.1:44885_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=1&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 301473 INFO  (OverseerThreadFactory-1424-thread-1-processing-n:127.0.0.1:39201_solr) [n:127.0.0.1:39201_solr    ] o.a.s.c.CreateCollectionCmd Create collection testcollection
   [junit4]   2> 301588 INFO  (OverseerStateUpdate-98464967727710222-127.0.0.1:39201_solr-n_0000000001) [n:127.0.0.1:39201_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:44885/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 301590 INFO  (OverseerStateUpdate-98464967727710222-127.0.0.1:39201_solr-n_0000000001) [n:127.0.0.1:39201_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:35085/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 301591 INFO  (OverseerStateUpdate-98464967727710222-127.0.0.1:39201_solr-n_0000000001) [n:127.0.0.1:39201_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n3",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:34069/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 301593 INFO  (OverseerStateUpdate-98464967727710222-127.0.0.1:39201_solr-n_0000000001) [n:127.0.0.1:39201_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:37699/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 301805 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node8&name=testcollection_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
   [junit4]   2> 301805 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 301806 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node5&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
   [junit4]   2> 301806 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node7&name=testcollection_shard2_replica_n3&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
   [junit4]   2> 301806 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 301806 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 301807 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node6&name=testcollection_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
   [junit4]   2> 301807 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 301914 INFO  (zkCallback-478-thread-1-processing-n:127.0.0.1:37699_solr) [n:127.0.0.1:37699_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 301914 INFO  (zkCallback-462-thread-1-processing-n:127.0.0.1:35085_solr) [n:127.0.0.1:35085_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 301914 INFO  (zkCallback-461-thread-1-processing-n:127.0.0.1:44885_solr) [n:127.0.0.1:44885_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 301914 INFO  (zkCallback-478-thread-2-processing-n:127.0.0.1:37699_solr) [n:127.0.0.1:37699_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 301914 INFO  (zkCallback-460-thread-1-processing-n:127.0.0.1:34069_solr) [n:127.0.0.1:34069_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 301919 INFO  (zkCallback-460-thread-2-processing-n:127.0.0.1:34069_solr) [n:127.0.0.1:34069_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 301919 INFO  (zkCallback-461-thread-2-processing-n:127.0.0.1:44885_solr) [n:127.0.0.1:44885_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 301920 INFO  (zkCallback-462-thread-2-processing-n:127.0.0.1:35085_solr) [n:127.0.0.1:35085_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 302821 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 302851 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 302853 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 302854 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 302875 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.s.IndexSchema [testcollection_shard2_replica_n3] Schema name=minimal
   [junit4]   2> 302878 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 302878 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard2_replica_n3' using configuration from collection testcollection, trusted=true
   [junit4]   2> 302879 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema [testcollection_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 302879 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.s.IndexSchema [testcollection_shard1_replica_n2] Schema name=minimal
   [junit4]   2> 302879 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.s.IndexSchema [testcollection_shard2_replica_n4] Schema name=minimal
   [junit4]   2> 302880 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34069.solr.core.testcollection.shard2.replica_n3' (registry 'solr.core.testcollection.shard2.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 302882 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 302882 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 302882 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n1' using configuration from collection testcollection, trusted=true
   [junit4]   2> 302882 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n2' using configuration from collection testcollection, trusted=true
   [junit4]   2> 302882 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35085.solr.core.testcollection.shard1.replica_n2' (registry 'solr.core.testcollection.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 302882 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44885.solr.core.testcollection.shard1.replica_n1' (registry 'solr.core.testcollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 302884 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 302884 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard2_replica_n4' using configuration from collection testcollection, trusted=true
   [junit4]   2> 302885 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37699.solr.core.testcollection.shard2.replica_n4' (registry 'solr.core.testcollection.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@47c215a2
   [junit4]   2> 302886 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 302886 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.c.SolrCore [[testcollection_shard2_replica_n3] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_24D1FBB0306F6AC6-001/tempDir-001/node2/testcollection_shard2_replica_n3], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_24D1FBB0306F6AC6-001/tempDir-001/node2/./testcollection_shard2_replica_n3/data/]
   [junit4]   2> 302886 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 302886 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [[testcollection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_24D1FBB0306F6AC6-001/tempDir-001/node3/testcollection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_24D1FBB0306F6AC6-001/tempDir-001/node3/./testcollection_shard1_replica_n1/data/]
   [junit4]   2> 302886 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 302886 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore [[testcollection_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_24D1FBB0306F6AC6-001/tempDir-001/node1/testcollection_shard1_replica_n2], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_24D1FBB0306F6AC6-001/tempDir-001/node1/./testcollection_shard1_replica_n2/data/]
   [junit4]   2> 302887 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 302887 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.c.SolrCore [[testcollection_shard2_replica_n4] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_24D1FBB0306F6AC6-001/tempDir-001/node6/testcollection_shard2_replica_n4], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCollectionsAPIViaSolrCloudCluster_24D1FBB0306F6AC6-001/tempDir-001/node6/./testcollection_shard2_replica_n4/data/]
   [junit4]   2> 302959 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 302960 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 302961 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 302961 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 302962 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 302962 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 302963 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 302963 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 302963 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@4b40eabf[testcollection_shard1_replica_n2] main]
   [junit4]   2> 302964 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 302964 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 302964 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 302964 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 302964 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 302964 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 302964 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 302964 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 302965 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 302965 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 302965 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 302965 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@e56319c[testcollection_shard2_replica_n3] main]
   [junit4]   2> 302966 INFO  (searcherExecutor-1433-thread-1-processing-n:127.0.0.1:35085_solr x:testcollection_shard1_replica_n2 s:shard1 c:testcollection r:core_node6) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore [testcollection_shard1_replica_n2] Registered new searcher Searcher@4b40eabf[testcollection_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 302966 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575439486108565504
   [junit4]   2> 302966 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@32319d2a[testcollection_shard2_replica_n4] main]
   [junit4]   2> 302967 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 302967 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 302967 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@140b0228[testcollection_shard1_replica_n1] main]
   [junit4]   2> 302967 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 302967 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 302967 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 302968 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 302982 INFO  (searcherExecutor-1434-thread-1-processing-n:127.0.0.1:37699_solr x:testcollection_shard2_replica_n4 s:shard2 c:testcollection r:core_node8) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.c.SolrCore [testcollection_shard2_replica_n4] Registered new searcher Searcher@32319d2a[testcollection_shard2_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 302982 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 302982 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575439486125342720
   [junit4]   2> 302983 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 302983 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 302983 INFO  (searcherExecutor-1431-thread-1-processing-n:127.0.0.1:34069_solr x:testcollection_shard2_replica_n3 s:shard2 c:testcollection r:core_node7) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.c.SolrCore [testcollection_shard2_replica_n3] Registered new searcher Searcher@e56319c[testcollection_shard2_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 302983 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575439486126391296
   [junit4]   2> 302984 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 302984 INFO  (searcherExecutor-1432-thread-1-processing-n:127.0.0.1:44885_solr x:testcollection_shard1_replica_n1 s:shard1 c:testcollection r:core_node5) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [testcollection_shard1_replica_n1] Registered new searcher Searcher@140b0228[testcollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 302984 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575439486127439872
   [junit4]   2> 302987 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 302987 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 302987 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34069/solr/testcollection_shard2_replica_n3/
   [junit4]   2> 302988 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.u.PeerSync PeerSync: core=testcollection_shard2_replica_n3 url=http://127.0.0.1:34069/solr START replicas=[http://127.0.0.1:37699/solr/testcollection_shard2_replica_n4/] nUpdates=100
   [junit4]   2> 302993 INFO  (qtp187003465-3022) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.c.S.Request [testcollection_shard2_replica_n4]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 303089 INFO  (zkCallback-462-thread-1-processing-n:127.0.0.1:35085_solr) [n:127.0.0.1:35085_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303089 INFO  (zkCallback-461-thread-1-processing-n:127.0.0.1:44885_solr) [n:127.0.0.1:44885_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303089 INFO  (zkCallback-460-thread-1-processing-n:127.0.0.1:34069_solr) [n:127.0.0.1:34069_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303089 INFO  (zkCallback-461-thread-2-processing-n:127.0.0.1:44885_solr) [n:127.0.0.1:44885_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303089 INFO  (zkCallback-460-thread-2-processing-n:127.0.0.1:34069_solr) [n:127.0.0.1:34069_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303089 INFO  (zkCallback-478-thread-1-processing-n:127.0.0.1:37699_solr) [n:127.0.0.1:37699_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303089 INFO  (zkCallback-462-thread-2-processing-n:127.0.0.1:35085_solr) [n:127.0.0.1:35085_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303089 INFO  (zkCallback-478-thread-2-processing-n:127.0.0.1:37699_solr) [n:127.0.0.1:37699_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303288 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.u.PeerSync PeerSync: core=testcollection_shard2_replica_n3 url=http://127.0.0.1:34069/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 303288 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 303288 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 303288 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 303290 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34069/solr/testcollection_shard2_replica_n3/ shard2
   [junit4]   2> 303391 INFO  (zkCallback-462-thread-1-processing-n:127.0.0.1:35085_solr) [n:127.0.0.1:35085_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303391 INFO  (zkCallback-461-thread-2-processing-n:127.0.0.1:44885_solr) [n:127.0.0.1:44885_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303391 INFO  (zkCallback-460-thread-1-processing-n:127.0.0.1:34069_solr) [n:127.0.0.1:34069_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303391 INFO  (zkCallback-462-thread-2-processing-n:127.0.0.1:35085_solr) [n:127.0.0.1:35085_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303392 INFO  (zkCallback-478-thread-2-processing-n:127.0.0.1:37699_solr) [n:127.0.0.1:37699_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303392 INFO  (zkCallback-478-thread-1-processing-n:127.0.0.1:37699_solr) [n:127.0.0.1:37699_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303391 INFO  (zkCallback-460-thread-2-processing-n:127.0.0.1:34069_solr) [n:127.0.0.1:34069_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303391 INFO  (zkCallback-461-thread-1-processing-n:127.0.0.1:44885_solr) [n:127.0.0.1:44885_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303441 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 303442 INFO  (qtp191499112-2899) [n:127.0.0.1:34069_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node7&name=testcollection_shard2_replica_n3&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1635
   [junit4]   2> 303483 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 303483 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 303483 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35085/solr/testcollection_shard1_replica_n2/
   [junit4]   2> 303484 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=testcollection_shard1_replica_n2 url=http://127.0.0.1:35085/solr START replicas=[http://127.0.0.1:44885/solr/testcollection_shard1_replica_n1/] nUpdates=100
   [junit4]   2> 303489 INFO  (qtp2024517883-2907) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.c.S.Request [testcollection_shard1_replica_n1]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 303543 INFO  (zkCallback-478-thread-2-processing-n:127.0.0.1:37699_solr) [n:127.0.0.1:37699_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303543 INFO  (zkCallback-478-thread-1-processing-n:127.0.0.1:37699_solr) [n:127.0.0.1:37699_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303543 INFO  (zkCallback-462-thread-1-processing-n:127.0.0.1:35085_solr) [n:127.0.0.1:35085_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303543 INFO  (zkCallback-460-thread-1-processing-n:127.0.0.1:34069_solr) [n:127.0.0.1:34069_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303543 INFO  (zkCallback-461-thread-2-processing-n:127.0.0.1:44885_solr) [n:127.0.0.1:44885_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303543 INFO  (zkCallback-460-thread-2-processing-n:127.0.0.1:34069_solr) [n:127.0.0.1:34069_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303543 INFO  (zkCallback-461-thread-1-processing-n:127.0.0.1:44885_solr) [n:127.0.0.1:44885_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303543 INFO  (zkCallback-462-thread-2-processing-n:127.0.0.1:35085_solr) [n:127.0.0.1:35085_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303784 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=testcollection_shard1_replica_n2 url=http://127.0.0.1:35085/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 303784 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 303784 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 303784 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 303786 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35085/solr/testcollection_shard1_replica_n2/ shard1
   [junit4]   2> 303887 INFO  (zkCallback-462-thread-1-processing-n:127.0.0.1:35085_solr) [n:127.0.0.1:35085_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303887 INFO  (zkCallback-462-thread-2-processing-n:127.0.0.1:35085_solr) [n:127.0.0.1:35085_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303887 INFO  (zkCallback-461-thread-2-processing-n:127.0.0.1:44885_solr) [n:127.0.0.1:44885_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303887 INFO  (zkCallback-460-thread-2-processing-n:127.0.0.1:34069_solr) [n:127.0.0.1:34069_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303887 INFO  (zkCallback-460-thread-1-processing-n:127.0.0.1:34069_solr) [n:127.0.0.1:34069_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303887 INFO  (zkCallback-478-thread-1-processing-n:127.0.0.1:37699_solr) [n:127.0.0.1:37699_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303887 INFO  (zkCallback-478-thread-2-processing-n:127.0.0.1:37699_solr) [n:127.0.0.1:37699_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303887 INFO  (zkCallback-461-thread-1-processing-n:127.0.0.1:44885_solr) [n:127.0.0.1:44885_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 303936 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 303937 INFO  (qtp1172646673-2902) [n:127.0.0.1:35085_solr c:testcollection s:shard1 r:core_node6 x:testcollection_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node6&name=testcollection_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=2130
   [junit4]   2> 303989 INFO  (qtp187003465-3020) [n:127.0.0.1:37699_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node8&name=testcollection_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=2183
   [junit4]   2> 303989 INFO  (qtp2024517883-2925) [n:127.0.0.1:44885_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCo

[...truncated too long message...]

hread-2-processing-n:127.0.0.1:34081_) [n:127.0.0.1:34081_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.c.SolrCore [collection1_shard2_replica_t45]  CLOSING SolrCore org.apache.solr.core.SolrCore@74f84890
   [junit4]   2> 1600969 INFO  (zkCallback-1815-thread-2-processing-n:127.0.0.1:34081_) [n:127.0.0.1:34081_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard2.replica_t45, tag=1962428560
   [junit4]   2> 1600972 INFO  (zkCallback-1815-thread-2-processing-n:127.0.0.1:34081_) [n:127.0.0.1:34081_ c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard2.leader, tag=1962428560
   [junit4]   2> 1600973 INFO  (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[24D1FBB0306F6AC6]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@b871b27{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1600973 INFO  (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[24D1FBB0306F6AC6]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1e8796e1{/,null,UNAVAILABLE}
   [junit4]   2> 1600973 ERROR (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[24D1FBB0306F6AC6]) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1600973 INFO  (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45499 45499
   [junit4]   2> 1606105 INFO  (Thread-7987) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45499 45499
   [junit4]   2> 1606106 WARN  (Thread-7987) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 6 /solr/aliases.json
   [junit4]   2> 4 /solr/security.json
   [junit4]   2> 4 /solr/configs/conf1
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 6 /solr/clusterprops.json
   [junit4]   2> 6 /solr/clusterstate.json
   [junit4]   2> 3 /solr/collections/forceleader_test_collection/state.json
   [junit4]   2> 3 /solr/collections/collection1/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 6 /solr/live_nodes
   [junit4]   2> 6 /solr/collections
   [junit4]   2> 3 /solr/overseer/queue
   [junit4]   2> 3 /solr/overseer/collection-queue-work
   [junit4]   2> 2 /solr/overseer/queue-work
   [junit4]   2>
   [junit4]   2> 1606106 WARN  (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.c.SocketProxy Closing 0 connections to: http://127.0.0.1:38021/, target: http://127.0.0.1:36277/
   [junit4]   2> 1606106 WARN  (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.c.SocketProxy Closing 1 connections to: http://127.0.0.1:34081/, target: http://127.0.0.1:43993/
   [junit4]   2> 1606106 WARN  (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.c.SocketProxy Closing 13 connections to: http://127.0.0.1:37079/, target: http://127.0.0.1:46199/
   [junit4]   2> 1606106 WARN  (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[24D1FBB0306F6AC6]) [    ] o.a.s.c.SocketProxy Closing 2 connections to: http://127.0.0.1:35913/, target: http://127.0.0.1:44083/
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ForceLeaderTest -Dtests.method=testReplicasInLIRNoLeader -Dtests.seed=24D1FBB0306F6AC6 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=it -Dtests.timezone=America/Indiana/Vevay -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 62.2s J0 | ForceLeaderTest.testReplicasInLIRNoLeader <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Doc with id=1 not found in http://127.0.0.1:37079/forceleader_test_collection due to: Path not found: /id; rsp={doc=null}
   [junit4]    > at __randomizedtesting.SeedInfo.seed([24D1FBB0306F6AC6:C246CF7009ED93A7]:0)
   [junit4]    > at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:603)
   [junit4]    > at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:556)
   [junit4]    > at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:142)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_24D1FBB0306F6AC6-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {multiDefault=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_t=PostingsFormat(name=Memory), id=PostingsFormat(name=LuceneVarGapFixedInterval), text=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene70), _version_=DocValuesFormat(name=Memory), intDefault=DocValuesFormat(name=Memory), id_i1=DocValuesFormat(name=Lucene70), range_facet_i_dv=DocValuesFormat(name=Memory), intDvoDefault=DocValuesFormat(name=Asserting), range_facet_l=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=406, maxMBSortInHeap=6.872523355047335, sim=RandomSimilarity(queryNorm=false): {}, locale=it, timezone=America/Indiana/Vevay
   [junit4]   2> NOTE: Linux 4.10.0-27-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=8,threads=1,free=259092576,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [TestConfigSetsAPIZkFailure, BlockCacheTest, TestSlowCompositeReaderWrapper, TestPullReplica, TestSystemCollAutoCreate, TestShardHandlerFactory, TestLMDirichletSimilarityFactory, TestIBSimilarityFactory, SolrCoreTest, SolrCloudExampleTest, TestWriterPerf, TestSchemaSimilarityResource, TestStressUserVersions, TestSolrCloudWithDelegationTokens, SparseHLLTest, IndexBasedSpellCheckerTest, SolrIndexSplitterTest, CoreAdminCreateDiscoverTest, TestImpersonationWithHadoopAuth, TestCSVResponseWriter, DistributedQueryComponentCustomSortTest, TestRealTimeGet, DocumentAnalysisRequestHandlerTest, PreAnalyzedUpdateProcessorTest, HttpSolrCallGetCoreTest, TestLegacyNumericUtils, CollectionStateFormat2Test, TestLeaderInitiatedRecoveryThread, CollectionsAPISolrJTest, TestCSVLoader, TestHighlightDedupGrouping, BJQParserTest, ConnectionManagerTest, TestCollectionsAPIViaSolrCloudCluster, TestHdfsUpdateLog, TestRandomCollapseQParserPlugin, TestReplicationHandler, TestDistributedSearch, TestExtendedDismaxParser, TestXmlQParserPlugin, TestLazyCores, TestCloudPivotFacet, TestFastOutputStream, DistributedQueryElevationComponentTest, DocExpirationUpdateProcessorFactoryTest, SolrMetricsIntegrationTest, BaseCdcrDistributedZkTest, DefaultValueUpdateProcessorTest, PreAnalyzedFieldManagedSchemaCloudTest, TestSubQueryTransformerCrossCore, TlogReplayBufferedWhileIndexingTest, TestRandomRequestDistribution, TestFieldCollectionResource, FacetPivotSmallTest, RequestHandlersTest, TestManagedResourceStorage, HdfsWriteToMultipleCollectionsTest, TestBinaryResponseWriter, TestSolrQueryResponse, ZkNodePropsTest, UUIDFieldTest, JavabinLoaderTest, TestQuerySenderNoQuery, TestIndexingPerformance, TestRTimerTree, TestConfigSets, ExternalFileFieldSortTest, TestMinMaxOnMultiValuedField, BigEndianAscendingWordSerializerTest, TestFaceting, PluginInfoTest, TestLRUStatsCache, SolrIndexConfigTest, TestStressLucene, TestPerFieldSimilarityWithDefaultOverride, TestManagedSchemaAPI, SyncSliceTest, TestPhraseSuggestions, TestGeoJSONResponseWriter, TestUninvertingReader, TestRecoveryHdfs, CdcrReplicationHandlerTest, TestHttpShardHandlerFactory, ShufflingReplicaListTransformerTest, SpellCheckCollatorTest, RuleEngineTest, SynonymTokenizerTest, ClassificationUpdateProcessorIntegrationTest, OpenExchangeRatesOrgProviderTest, TestComponentsName, SolrShardReporterTest, TestMultiWordSynonyms, HdfsRecoveryZkTest, TestBinaryField, DocValuesMissingTest, ChaosMonkeyNothingIsSafeTest, PKIAuthenticationIntegrationTest, FieldMutatingUpdateProcessorTest, InfoHandlerTest, TestInitParams, SuggesterWFSTTest, TestUseDocValuesAsStored2, OverseerTaskQueueTest, CursorMarkTest, CleanupOldIndexTest, TestDistribDocBasedVersion, DistributedSpellCheckComponentTest, HdfsDirectoryTest, TestFieldTypeResource, LeaderInitiatedRecoveryOnCommitTest, TestPayloadCheckQParserPlugin, TestGraphTermsQParserPlugin, TestNumericRangeQuery32, ZkStateReaderTest, TestMacros, XmlUpdateRequestHandlerTest, AtomicUpdateProcessorFactoryTest, RestartWhileUpdatingTest, CollectionsAPIDistributedZkTest, BasicZkTest, FullSolrCloudDistribCmdsTest, TestZkChroot, TestRandomDVFaceting, TestJoin, SpellCheckComponentTest, BadIndexSchemaTest, ConvertedLegacyTest, TestSort, TestFunctionQuery, BasicFunctionalityTest, DirectUpdateHandlerTest, TestBadConfig, SoftAutoCommitTest, TestLuceneMatchVersion, TestPerFieldSimilarity, TestBM25SimilarityFactory, TimeZoneUtilsTest, DateMathParserTest, TestSuggestSpellingConverter, TestSolrJ, TestUtils, SolrTestCaseJ4Test, TestCrossCoreJoin, TestDistributedMissingSort, ActionThrottleTest, AssignBackwardCompatibilityTest, CollectionReloadTest, CollectionTooManyReplicasTest, CollectionsAPIAsyncDistributedZkTest, DistribCursorPagingTest, ForceLeaderTest]
   [junit4] Completed [525/729 (2!)] on J0 in 130.43s, 3 tests, 1 failure, 1 skipped <<< FAILURES!

[...truncated 43935 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Loading...