Solr Document Update issues

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Solr Document Update issues

aparana bhatt
Hi ,

I have been facing lot of issues in using solr update functionality .
Multitude of requests respond with










* org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://192.169.33.86/solr/cms
<http://192.169.33.86/solr/cms>: Expected mime type
application/octet-stream but got text/html. <!DOCTYPE HTML PUBLIC
"-//IETF//DTD HTML 2.0//EN"><html><head><title>502 Proxy
Error</title></head><body><h1>Proxy Error</h1><p>The proxy server received
an invalid^Mresponse from an upstream server.<br />^MThe proxy server could
not handle the request <em><a
href="/solr/cms/update">POST&nbsp;/solr/cms/update</a></em>.<p>Reason:
<strong>Error reading from remote server</strong></p></p></body></html>*

Used solr version -> 6.5.0  Type -> master/Slave config
Error in solr.log ->



























*2020-10-07 05:43:50.639 WARN  (qtp142261320-27831) [   x:cms]
o.a.s.c.SolrCore slow: [cms]  webapp=/solr path=/update
params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}
status=0 QTime=443272020-10-07 05:43:50.640 WARN  (qtp142261320-27837) [
x:cms] o.a.s.u.DefaultSolrCoreState WARNING - Dangerous
interruptjava.lang.InterruptedException        at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326)
      at
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.tryLock(ReentrantReadWriteLock.java:871)
      at
org.apache.solr.update.DefaultSolrCoreState.lock(DefaultSolrCoreState.java:167)
      at
org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:112)
      at
org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:618)
      at
org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:93)
      at
org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:68)
      at
org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1895)
      at
org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1872)
      at
org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:68)
      at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:72)
      at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
      at org.apache.solr.core.SolrCore.execute(SolrCore.java:2440)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
  at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
      at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
      at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
      at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
      at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
      at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
      at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
      at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)*



The rate of update query on master solr is 6 request per min only .
Solr also slows down and search becomes really slow .
I don't understand where to look for an issue .
I have tried to check various parameters in update request , if I do
softcommit=true and commit =false then updates do not reflect , so i have
set below options ->

UpdateRequest updateRequest = new UpdateRequest();
          updateRequest.setAction( UpdateRequest.ACTION.COMMIT, true,
true);
waitsearcher=true ,
waitflush=true .

I do not get what is causing the issue . Kindly suggest .
Also I could not find much help from internet about given issues as well .


--
Regards

Aparana Bhatt
Reply | Threaded
Open this post in threaded view
|

Re: Solr Document Update issues

Radu Gheorghe
Hi,

I wouldn’t commit on every update. The general practice is to use autoCommit and autoSoftCommit, so this work is done in background depending on how quickly you want data persisted and available for search: https://lucene.apache.org/solr/guide/6_6/updatehandlers-in-solrconfig.html#UpdateHandlersinSolrConfig-Commits

Best regards,
Radu
--
Sematext Cloud - Full Stack Observability - https://sematext.com
Solr and Elasticsearch Consulting, Training and Production Support

> On 13 Oct 2020, at 07:18, aparana bhatt <[hidden email]> wrote:
>
> Hi ,
>
> I have been facing lot of issues in using solr update functionality .
> Multitude of requests respond with
>
>
>
>
>
>
>
>
>
>
> * org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
> Error from server at http://192.169.33.86/solr/cms
> <http://192.169.33.86/solr/cms>: Expected mime type
> application/octet-stream but got text/html. <!DOCTYPE HTML PUBLIC
> "-//IETF//DTD HTML 2.0//EN"><html><head><title>502 Proxy
> Error</title></head><body><h1>Proxy Error</h1><p>The proxy server received
> an invalid^Mresponse from an upstream server.<br />^MThe proxy server could
> not handle the request <em><a
> href="/solr/cms/update">POST&nbsp;/solr/cms/update</a></em>.<p>Reason:
> <strong>Error reading from remote server</strong></p></p></body></html>*
>
> Used solr version -> 6.5.0  Type -> master/Slave config
> Error in solr.log ->
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *2020-10-07 05:43:50.639 WARN  (qtp142261320-27831) [   x:cms]
> o.a.s.c.SolrCore slow: [cms]  webapp=/solr path=/update
> params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}
> status=0 QTime=443272020-10-07 05:43:50.640 WARN  (qtp142261320-27837) [
> x:cms] o.a.s.u.DefaultSolrCoreState WARNING - Dangerous
> interruptjava.lang.InterruptedException        at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326)
>      at
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.tryLock(ReentrantReadWriteLock.java:871)
>      at
> org.apache.solr.update.DefaultSolrCoreState.lock(DefaultSolrCoreState.java:167)
>      at
> org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:112)
>      at
> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:618)
>      at
> org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:93)
>      at
> org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:68)
>      at
> org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1895)
>      at
> org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1872)
>      at
> org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:68)
>      at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:72)
>      at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
>      at org.apache.solr.core.SolrCore.execute(SolrCore.java:2440)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
>  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
>  at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
>      at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
>      at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
>      at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
>      at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>      at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
>      at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
>      at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
>      at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)*
>
>
>
> The rate of update query on master solr is 6 request per min only .
> Solr also slows down and search becomes really slow .
> I don't understand where to look for an issue .
> I have tried to check various parameters in update request , if I do
> softcommit=true and commit =false then updates do not reflect , so i have
> set below options ->
>
> UpdateRequest updateRequest = new UpdateRequest();
>          updateRequest.setAction( UpdateRequest.ACTION.COMMIT, true,
> true);
> waitsearcher=true ,
> waitflush=true .
>
> I do not get what is causing the issue . Kindly suggest .
> Also I could not find much help from internet about given issues as well .
>
>
> --
> Regards
>
> Aparana Bhatt