[jira] [Commented] (SOLR-11638) CloudSolrClientTest periodic failures

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-11638) CloudSolrClientTest periodic failures

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/SOLR-11638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16250534#comment-16250534 ]

David Smiley commented on SOLR-11638:
-------------------------------------

Thanks guys!
BTW if a recently closed issue introduces failures, no need to create additional JIRA issues. Just reopen.

> CloudSolrClientTest periodic failures
> -------------------------------------
>
>                 Key: SOLR-11638
>                 URL: https://issues.apache.org/jira/browse/SOLR-11638
>             Project: Solr
>          Issue Type: Bug
>      Security Level: Public(Default Security Level. Issues are Public)
>          Components: SolrJ, Tests
>    Affects Versions: master (8.0)
>            Reporter: Jason Gerlowski
>            Assignee: Shalin Shekhar Mangar
>             Fix For: 7.2, master (8.0)
>
>         Attachments: SOLR-11638.patch
>
>
> The test-randomization recently-added as a part of SOLR-11507 has caused {{CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale}} to fail semi-regularly on master.  The test only succeeds for me on 3 out of 10 test runs.  The test fails with the message:
> {code}
>    [junit4]   2> 14848 ERROR (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[64E89FBB977E15AA]) [    ] o.a.s.c.s.i.CloudSolrClient Request to collection [stale_state_test_col] failed due to (404) org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38925/solr/stale_state_test_col_shard1_replica_n1: Expected mime type application/octet-stream but got text/html. <html>
>    [junit4]   2> <head>
>    [junit4]   2> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
>    [junit4]   2> <title>Error 404 </title>
>    [junit4]   2> </head>
>    [junit4]   2> <body>
>    [junit4]   2> <h2>HTTP ERROR: 404</h2>
>    [junit4]   2> <p>Problem accessing /solr/stale_state_test_col_shard1_replica_n1/update. Reason:
>    [junit4]   2> <pre>    Can not find: /solr/stale_state_test_col_shard1_replica_n1/update</pre></p>
>    [junit4]   2> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/>
>    [junit4]   2> </body>
>    [junit4]   2> </html>
>    [junit4]   2> , retry? 0
>    [junit4]   2> 14851 INFO  (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[64E89FBB977E15AA]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testRetryUpdatesWhenClusterStateIsStale
>    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CloudSolrClientTest -Dtests.method=testRetryUpdatesWhenClusterStateIsStale -Dtests.seed=64E89FBB977E15AA -Dtests.slow=true -Dtests.locale=es-VE -Dtests.timezone=Indian/Chagos -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>    [junit4] ERROR   5.86s | CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale <<<
>    [junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38925/solr/stale_state_test_col_shard1_replica_n1: Expected mime type application/octet-stream but got text/html. <html>
>    [junit4]    > <head>
>    [junit4]    > <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
>    [junit4]    > <title>Error 404 </title>
>    [junit4]    > </head>
>    [junit4]    > <body>
>    [junit4]    > <h2>HTTP ERROR: 404</h2>
>    [junit4]    > <p>Problem accessing /solr/stale_state_test_col_shard1_replica_n1/update. Reason:
>    [junit4]    > <pre>    Can not find: /solr/stale_state_test_col_shard1_replica_n1/update</pre></p>
>    [junit4]    > <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/>
>    [junit4]    > </body>
>    [junit4]    > </html>
>    [junit4]    > at __randomizedtesting.SeedInfo.seed([64E89FBB977E15AA:D0D9075374976386]:0)
>    [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
>    [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
>    [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
>    [junit4]    > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
>    [junit4]    > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
>    [junit4]    > at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:559)
>    [junit4]    > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1016)
>    [junit4]    > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
>    [junit4]    > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
>    [junit4]    > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
>    [junit4]    > at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
>    [junit4]    > at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale(CloudSolrClientTest.java:844)
>    [junit4]    > at java.lang.Thread.run(Thread.java:748)
> {code}
> After some digging, it looks like the issue is that testRetryUpdatesWhenClusterStateIsStale implicitly relies on {{directUpdatesToLeadersOnly}}, and {{parallelUpdates}}, which are now randomized when using SolrTestCaseJ4's CloudSolrClient creation helpers.
> Attached is a patch ensuring that testRetryUpdatesWhenClusterStateIsStale insists on those two update-related properties, instead of taking the randomized defaults.  Without the patch, this test passes maybe 5 out of twenty times. With the patch, it passes consistently (20 out of 20 runs).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]