2 Async exceptions during distributed update

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

2 Async exceptions during distributed update

Fiz Ahmed
Hi Solr Experts,

SOLR Version : 6.6.2

OS – Linux 3.1.2

JDK – 1.8



Shard – 16 – All are active.

Xms – 16 gb

Xmx – 16 gb



Schema fields count – 91

Dynamic fields – 83.



There are multiple threads sending products to solr. tested with batch size
per thread 50 & 100, it worked fine without error and if the batch size is
1000 the following error occurs.

there are 64 cores that are sending batches concurrently.



*I am getting the following error when the batch size is 1000. Please
advise.*



2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY] -
{collection=c:ducts, core=x:ducts_shard15_replica1,
node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30, shard=s:shard15}
-
org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
2 Async exceptions during distributed update:

10.YYY.40.62:8983 failed to respond

10.YYY.40.62:8983 failed to respond



2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY] -
{collection=c:ducts, core=x:ducts_shard7_replica1,
node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29, shard=s:shard7}
-
org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
Async exception during distributed update: 10.YYY.40.81:8983 failed to
respond





2019-11-14T19:36:11,599 - ERROR
[updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
<http://10.148.40.68:8983/solr/products_shard11_replica2> r:core_node26
n:10.YYY.40.68:8983_solr c:ducts
s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
- {ConcurrentUpdateSolrClient.url=
http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
<http://10.148.40.68:8983/solr/products_shard11_replica2>,
collection=c:ducts, core=x:ducts_shard3_replica2,
node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26, shard=s:shard3}
- error

org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to respond

        at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
~[httpclient-4.4.1.jar:4.4.1]

        at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
~[httpclient-4.4.1.jar:4.4.1]

        at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
~[httpcore-4.4.1.jar:4.4.1]





2019-11-14T19:36:14,567 - ERROR
[updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
<http://10.148.40.62:8983/solr/products_shard2_replica1> r:core_node25
n:10.YYY.40.68:8983_solr c:ducts
s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
- {ConcurrentUpdateSolrClient.url=
http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
<http://10.148.40.62:8983/solr/products_shard2_replica1>,
collection=c:ducts, core=x:ducts_shard11_replica2,
node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
- error

java.net.SocketException: Broken pipe (Write failed)

        at java.net.SocketOutputStream.socketWrite0(Native Method)
~[?:1.8.0_232]

        at
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
~[?:1.8.0_232]

        at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
~[?:1.8.0_232]





2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159] -
{collection=c:ducts, core=x:ducts_shard11_replica2,
node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
-
null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
Async exception during distributed update: Broken pipe (Write failed)

       at
org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)

        at
org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)




Thanks

Fiz..
Reply | Threaded
Open this post in threaded view
|

Re: 2 Async exceptions during distributed update

Jörn Franke
Do you use a updateprocess factory? How does it look like?

What is the physical memory size and CPU?
What do you mean by “there are 64 cores sending concurrently?” An application has 64 threats that send concurrently those updates?

> Am 15.11.2019 um 02:14 schrieb Fiz N <[hidden email]>:
>
> Hi Solr Experts,
>
> SOLR Version : 6.6.2
>
> OS – Linux 3.1.2
>
> JDK – 1.8
>
>
>
> Shard – 16 – All are active.
>
> Xms – 16 gb
>
> Xmx – 16 gb
>
>
>
> Schema fields count – 91
>
> Dynamic fields – 83.
>
>
>
> There are multiple threads sending products to solr. tested with batch size
> per thread 50 & 100, it worked fine without error and if the batch size is
> 1000 the following error occurs.
>
> there are 64 cores that are sending batches concurrently.
>
>
>
> *I am getting the following error when the batch size is 1000. Please
> advise.*
>
>
>
> 2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY] -
> {collection=c:ducts, core=x:ducts_shard15_replica1,
> node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30, shard=s:shard15}
> -
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> 2 Async exceptions during distributed update:
>
> 10.YYY.40.62:8983 failed to respond
>
> 10.YYY.40.62:8983 failed to respond
>
>
>
> 2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY] -
> {collection=c:ducts, core=x:ducts_shard7_replica1,
> node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29, shard=s:shard7}
> -
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> Async exception during distributed update: 10.YYY.40.81:8983 failed to
> respond
>
>
>
>
>
> 2019-11-14T19:36:11,599 - ERROR
> [updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
> http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
> <http://10.148.40.68:8983/solr/products_shard11_replica2> r:core_node26
> n:10.YYY.40.68:8983_solr c:ducts
> s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
> - {ConcurrentUpdateSolrClient.url=
> http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
> <http://10.148.40.68:8983/solr/products_shard11_replica2>,
> collection=c:ducts, core=x:ducts_shard3_replica2,
> node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26, shard=s:shard3}
> - error
>
> org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to respond
>
>        at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
> ~[httpclient-4.4.1.jar:4.4.1]
>
>        at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
> ~[httpclient-4.4.1.jar:4.4.1]
>
>        at
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
> ~[httpcore-4.4.1.jar:4.4.1]
>
>
>
>
>
> 2019-11-14T19:36:14,567 - ERROR
> [updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
> http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
> <http://10.148.40.62:8983/solr/products_shard2_replica1> r:core_node25
> n:10.YYY.40.68:8983_solr c:ducts
> s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
> - {ConcurrentUpdateSolrClient.url=
> http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
> <http://10.148.40.62:8983/solr/products_shard2_replica1>,
> collection=c:ducts, core=x:ducts_shard11_replica2,
> node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
> - error
>
> java.net.SocketException: Broken pipe (Write failed)
>
>        at java.net.SocketOutputStream.socketWrite0(Native Method)
> ~[?:1.8.0_232]
>
>        at
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
> ~[?:1.8.0_232]
>
>        at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
> ~[?:1.8.0_232]
>
>
>
>
>
> 2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159] -
> {collection=c:ducts, core=x:ducts_shard11_replica2,
> node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
> -
> null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> Async exception during distributed update: Broken pipe (Write failed)
>
>       at
> org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
>
>        at
> org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
>
>
>
>
> Thanks
>
> Fiz..
Reply | Threaded
Open this post in threaded view
|

Re: 2 Async exceptions during distributed update

Fiz Ahmed
Thanks for your response.

<directoryFactory name="DirectoryFactory"class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>

 

attaching the screenshot of physical memory and cpu.

Please let me know your thoughts on the below issue.



On Fri, Nov 15, 2019 at 2:18 AM Jörn Franke <[hidden email]> wrote:
Do you use a updateprocess factory? How does it look like?

What is the physical memory size and CPU?
What do you mean by “there are 64 cores sending concurrently?” An application has 64 threats that send concurrently those updates?

> Am 15.11.2019 um 02:14 schrieb Fiz N <[hidden email]>:
>
> Hi Solr Experts,
>
> SOLR Version : 6.6.2
>
> OS – Linux 3.1.2
>
> JDK – 1.8
>
>
>
> Shard – 16 – All are active.
>
> Xms – 16 gb
>
> Xmx – 16 gb
>
>
>
> Schema fields count – 91
>
> Dynamic fields – 83.
>
>
>
> There are multiple threads sending products to solr. tested with batch size
> per thread 50 & 100, it worked fine without error and if the batch size is
> 1000 the following error occurs.
>
> there are 64 cores that are sending batches concurrently.
>
>
>
> *I am getting the following error when the batch size is 1000. Please
> advise.*
>
>
>
> 2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY] -
> {collection=c:ducts, core=x:ducts_shard15_replica1,
> node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30, shard=s:shard15}
> -
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> 2 Async exceptions during distributed update:
>
> 10.YYY.40.62:8983 failed to respond
>
> 10.YYY.40.62:8983 failed to respond
>
>
>
> 2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY] -
> {collection=c:ducts, core=x:ducts_shard7_replica1,
> node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29, shard=s:shard7}
> -
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> Async exception during distributed update: 10.YYY.40.81:8983 failed to
> respond
>
>
>
>
>
> 2019-11-14T19:36:11,599 - ERROR
> [updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
> http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
> <http://10.148.40.68:8983/solr/products_shard11_replica2> r:core_node26
> n:10.YYY.40.68:8983_solr c:ducts
> s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
> - {ConcurrentUpdateSolrClient.url=
> http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
> <http://10.148.40.68:8983/solr/products_shard11_replica2>,
> collection=c:ducts, core=x:ducts_shard3_replica2,
> node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26, shard=s:shard3}
> - error
>
> org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to respond
>
>        at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
> ~[httpclient-4.4.1.jar:4.4.1]
>
>        at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
> ~[httpclient-4.4.1.jar:4.4.1]
>
>        at
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
> ~[httpcore-4.4.1.jar:4.4.1]
>
>
>
>
>
> 2019-11-14T19:36:14,567 - ERROR
> [updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
> http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
> <http://10.148.40.62:8983/solr/products_shard2_replica1> r:core_node25
> n:10.YYY.40.68:8983_solr c:ducts
> s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
> - {ConcurrentUpdateSolrClient.url=
> http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
> <http://10.148.40.62:8983/solr/products_shard2_replica1>,
> collection=c:ducts, core=x:ducts_shard11_replica2,
> node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
> - error
>
> java.net.SocketException: Broken pipe (Write failed)
>
>        at java.net.SocketOutputStream.socketWrite0(Native Method)
> ~[?:1.8.0_232]
>
>        at
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
> ~[?:1.8.0_232]
>
>        at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
> ~[?:1.8.0_232]
>
>
>
>
>
> 2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159] -
> {collection=c:ducts, core=x:ducts_shard11_replica2,
> node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
> -
> null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> Async exception during distributed update: Broken pipe (Write failed)
>
>       at
> org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
>
>        at
> org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
>
>
>
>
> Thanks
>
> Fiz..
Reply | Threaded
Open this post in threaded view
|

Re: 2 Async exceptions during distributed update

Fiz Ahmed
Hi Solr Experts,

Do you have any thoughts on the below issue ?

On Fri, Nov 15, 2019 at 11:33 AM Fiz N <[hidden email]> wrote:

> Thanks for your response.
>
> <directoryFactory name="DirectoryFactory"class=
> "${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
>
>
>
> attaching the screenshot of physical memory and cpu.
>
> Please let me know your thoughts on the below issue.
>
>
>
> On Fri, Nov 15, 2019 at 2:18 AM Jörn Franke <[hidden email]> wrote:
>
>> Do you use a updateprocess factory? How does it look like?
>>
>> What is the physical memory size and CPU?
>> What do you mean by “there are 64 cores sending concurrently?” An
>> application has 64 threats that send concurrently those updates?
>>
>> > Am 15.11.2019 um 02:14 schrieb Fiz N <[hidden email]>:
>> >
>> > Hi Solr Experts,
>> >
>> > SOLR Version : 6.6.2
>> >
>> > OS – Linux 3.1.2
>> >
>> > JDK – 1.8
>> >
>> >
>> >
>> > Shard – 16 – All are active.
>> >
>> > Xms – 16 gb
>> >
>> > Xmx – 16 gb
>> >
>> >
>> >
>> > Schema fields count – 91
>> >
>> > Dynamic fields – 83.
>> >
>> >
>> >
>> > There are multiple threads sending products to solr. tested with batch
>> size
>> > per thread 50 & 100, it worked fine without error and if the batch size
>> is
>> > 1000 the following error occurs.
>> >
>> > there are 64 cores that are sending batches concurrently.
>> >
>> >
>> >
>> > *I am getting the following error when the batch size is 1000. Please
>> > advise.*
>> >
>> >
>> >
>> > 2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard15_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30,
>> shard=s:shard15}
>> > -
>> >
>> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > 2 Async exceptions during distributed update:
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> >
>> >
>> > 2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard7_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29,
>> shard=s:shard7}
>> > -
>> >
>> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: 10.YYY.40.81:8983 failed to
>> > respond
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:11,599 - ERROR
>> > [updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
>> > http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2> r:core_node26
>> > n:10.YYY.40.68:8983_solr c:ducts
>> >
>> s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131
>> ]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2>,
>> > collection=c:ducts, core=x:ducts_shard3_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26,
>> shard=s:shard3}
>> > - error
>> >
>> > org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to
>> respond
>> >
>> >        at
>> >
>> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> >        at
>> >
>> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> >        at
>> > org.apache.http.impl.io
>> .AbstractMessageParser.parse(AbstractMessageParser.java:261)
>> > ~[httpcore-4.4.1.jar:4.4.1]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:14,567 - ERROR
>> > [updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
>> > http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1> r:core_node25
>> > n:10.YYY.40.68:8983_solr c:ducts
>> >
>> s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131
>> ]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1>,
>> > collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25,
>> shard=s:shard11}
>> > - error
>> >
>> > java.net.SocketException: Broken pipe (Write failed)
>> >
>> >        at java.net.SocketOutputStream.socketWrite0(Native Method)
>> > ~[?:1.8.0_232]
>> >
>> >        at
>> > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
>> > ~[?:1.8.0_232]
>> >
>> >        at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
>> > ~[?:1.8.0_232]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159] -
>> > {collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25,
>> shard=s:shard11}
>> > -
>> >
>> null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: Broken pipe (Write failed)
>> >
>> >       at
>> >
>> org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
>> >
>> >        at
>> >
>> org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
>> >
>> >
>> >
>> >
>> > Thanks
>> >
>> > Fiz..
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: 2 Async exceptions during distributed update

Jörn Franke
In reply to this post by Fiz Ahmed
No screenshot attached. The Apache mailservers filter attachments. Can you
please provide an external link.

On Fri, Nov 15, 2019 at 5:34 PM Fiz N <[hidden email]> wrote:

> Thanks for your response.
>
> <directoryFactory name="DirectoryFactory"class=
> "${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
>
>
>
> attaching the screenshot of physical memory and cpu.
>
> Please let me know your thoughts on the below issue.
>
>
>
> On Fri, Nov 15, 2019 at 2:18 AM Jörn Franke <[hidden email]> wrote:
>
>> Do you use a updateprocess factory? How does it look like?
>>
>> What is the physical memory size and CPU?
>> What do you mean by “there are 64 cores sending concurrently?” An
>> application has 64 threats that send concurrently those updates?
>>
>> > Am 15.11.2019 um 02:14 schrieb Fiz N <[hidden email]>:
>> >
>> > Hi Solr Experts,
>> >
>> > SOLR Version : 6.6.2
>> >
>> > OS – Linux 3.1.2
>> >
>> > JDK – 1.8
>> >
>> >
>> >
>> > Shard – 16 – All are active.
>> >
>> > Xms – 16 gb
>> >
>> > Xmx – 16 gb
>> >
>> >
>> >
>> > Schema fields count – 91
>> >
>> > Dynamic fields – 83.
>> >
>> >
>> >
>> > There are multiple threads sending products to solr. tested with batch
>> size
>> > per thread 50 & 100, it worked fine without error and if the batch size
>> is
>> > 1000 the following error occurs.
>> >
>> > there are 64 cores that are sending batches concurrently.
>> >
>> >
>> >
>> > *I am getting the following error when the batch size is 1000. Please
>> > advise.*
>> >
>> >
>> >
>> > 2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard15_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30,
>> shard=s:shard15}
>> > -
>> >
>> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > 2 Async exceptions during distributed update:
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> >
>> >
>> > 2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard7_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29,
>> shard=s:shard7}
>> > -
>> >
>> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: 10.YYY.40.81:8983 failed to
>> > respond
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:11,599 - ERROR
>> > [updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
>> > http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2> r:core_node26
>> > n:10.YYY.40.68:8983_solr c:ducts
>> >
>> s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131
>> ]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2>,
>> > collection=c:ducts, core=x:ducts_shard3_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26,
>> shard=s:shard3}
>> > - error
>> >
>> > org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to
>> respond
>> >
>> >        at
>> >
>> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> >        at
>> >
>> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> >        at
>> > org.apache.http.impl.io
>> .AbstractMessageParser.parse(AbstractMessageParser.java:261)
>> > ~[httpcore-4.4.1.jar:4.4.1]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:14,567 - ERROR
>> > [updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
>> > http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1> r:core_node25
>> > n:10.YYY.40.68:8983_solr c:ducts
>> >
>> s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131
>> ]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1>,
>> > collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25,
>> shard=s:shard11}
>> > - error
>> >
>> > java.net.SocketException: Broken pipe (Write failed)
>> >
>> >        at java.net.SocketOutputStream.socketWrite0(Native Method)
>> > ~[?:1.8.0_232]
>> >
>> >        at
>> > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
>> > ~[?:1.8.0_232]
>> >
>> >        at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
>> > ~[?:1.8.0_232]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159] -
>> > {collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25,
>> shard=s:shard11}
>> > -
>> >
>> null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: Broken pipe (Write failed)
>> >
>> >       at
>> >
>> org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
>> >
>> >        at
>> >
>> org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
>> >
>> >
>> >
>> >
>> > Thanks
>> >
>> > Fiz..
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: 2 Async exceptions during distributed update

Jörn Franke
In reply to this post by Fiz Ahmed
Can you please provide the whole update chain ? Below is the directoryFactory and not the update processor chain.

> Am 15.11.2019 um 17:34 schrieb Fiz N <[hidden email]>:
>
> 
> Thanks for your response.
>
> <directoryFactory name="DirectoryFactory"class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>
>  
> attaching the screenshot of physical memory and cpu.
> Please let me know your thoughts on the below issue.
>
>
>> On Fri, Nov 15, 2019 at 2:18 AM Jörn Franke <[hidden email]> wrote:
>> Do you use a updateprocess factory? How does it look like?
>>
>> What is the physical memory size and CPU?
>> What do you mean by “there are 64 cores sending concurrently?” An application has 64 threats that send concurrently those updates?
>>
>> > Am 15.11.2019 um 02:14 schrieb Fiz N <[hidden email]>:
>> >
>> > Hi Solr Experts,
>> >
>> > SOLR Version : 6.6.2
>> >
>> > OS – Linux 3.1.2
>> >
>> > JDK – 1.8
>> >
>> >
>> >
>> > Shard – 16 – All are active.
>> >
>> > Xms – 16 gb
>> >
>> > Xmx – 16 gb
>> >
>> >
>> >
>> > Schema fields count – 91
>> >
>> > Dynamic fields – 83.
>> >
>> >
>> >
>> > There are multiple threads sending products to solr. tested with batch size
>> > per thread 50 & 100, it worked fine without error and if the batch size is
>> > 1000 the following error occurs.
>> >
>> > there are 64 cores that are sending batches concurrently.
>> >
>> >
>> >
>> > *I am getting the following error when the batch size is 1000. Please
>> > advise.*
>> >
>> >
>> >
>> > 2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard15_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30, shard=s:shard15}
>> > -
>> > org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > 2 Async exceptions during distributed update:
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> >
>> >
>> > 2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard7_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29, shard=s:shard7}
>> > -
>> > org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: 10.YYY.40.81:8983 failed to
>> > respond
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:11,599 - ERROR
>> > [updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
>> > http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2> r:core_node26
>> > n:10.YYY.40.68:8983_solr c:ducts
>> > s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2>,
>> > collection=c:ducts, core=x:ducts_shard3_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26, shard=s:shard3}
>> > - error
>> >
>> > org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to respond
>> >
>> >        at
>> > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> >        at
>> > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> >        at
>> > org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
>> > ~[httpcore-4.4.1.jar:4.4.1]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:14,567 - ERROR
>> > [updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
>> > http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1> r:core_node25
>> > n:10.YYY.40.68:8983_solr c:ducts
>> > s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1>,
>> > collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
>> > - error
>> >
>> > java.net.SocketException: Broken pipe (Write failed)
>> >
>> >        at java.net.SocketOutputStream.socketWrite0(Native Method)
>> > ~[?:1.8.0_232]
>> >
>> >        at
>> > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
>> > ~[?:1.8.0_232]
>> >
>> >        at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
>> > ~[?:1.8.0_232]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159] -
>> > {collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
>> > -
>> > null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: Broken pipe (Write failed)
>> >
>> >       at
>> > org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
>> >
>> >        at
>> > org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
>> >
>> >
>> >
>> >
>> > Thanks
>> >
>> > Fiz..
Reply | Threaded
Open this post in threaded view
|

Re: 2 Async exceptions during distributed update

Fiz Ahmed
Thanks Jorn on getting back to me. Please find attached memory/cpu utilization and complete update process chain.
Please let me know your thoughts.

On Sat, Nov 16, 2019 at 9:09 AM Jörn Franke <[hidden email]> wrote:
Can you please provide the whole update chain ? Below is the directoryFactory and not the update processor chain.

> Am 15.11.2019 um 17:34 schrieb Fiz N <[hidden email]>:
>
> 
> Thanks for your response.
>
> <directoryFactory name="DirectoryFactory"class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>

> attaching the screenshot of physical memory and cpu.
> Please let me know your thoughts on the below issue.
>
>
>> On Fri, Nov 15, 2019 at 2:18 AM Jörn Franke <[hidden email]> wrote:
>> Do you use a updateprocess factory? How does it look like?
>>
>> What is the physical memory size and CPU?
>> What do you mean by “there are 64 cores sending concurrently?” An application has 64 threats that send concurrently those updates?
>>
>> > Am 15.11.2019 um 02:14 schrieb Fiz N <[hidden email]>:
>> >
>> > Hi Solr Experts,
>> >
>> > SOLR Version : 6.6.2
>> >
>> > OS – Linux 3.1.2
>> >
>> > JDK – 1.8
>> >
>> >
>> >
>> > Shard – 16 – All are active.
>> >
>> > Xms – 16 gb
>> >
>> > Xmx – 16 gb
>> >
>> >
>> >
>> > Schema fields count – 91
>> >
>> > Dynamic fields – 83.
>> >
>> >
>> >
>> > There are multiple threads sending products to solr. tested with batch size
>> > per thread 50 & 100, it worked fine without error and if the batch size is
>> > 1000 the following error occurs.
>> >
>> > there are 64 cores that are sending batches concurrently.
>> >
>> >
>> >
>> > *I am getting the following error when the batch size is 1000. Please
>> > advise.*
>> >
>> >
>> >
>> > 2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard15_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30, shard=s:shard15}
>> > -
>> > org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > 2 Async exceptions during distributed update:
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> >
>> >
>> > 2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard7_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29, shard=s:shard7}
>> > -
>> > org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: 10.YYY.40.81:8983 failed to
>> > respond
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:11,599 - ERROR
>> > [updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
>> > http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2> r:core_node26
>> > n:10.YYY.40.68:8983_solr c:ducts
>> > s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2>,
>> > collection=c:ducts, core=x:ducts_shard3_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26, shard=s:shard3}
>> > - error
>> >
>> > org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to respond
>> >
>> >        at
>> > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> >        at
>> > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> >        at
>> > org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
>> > ~[httpcore-4.4.1.jar:4.4.1]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:14,567 - ERROR
>> > [updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
>> > http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1> r:core_node25
>> > n:10.YYY.40.68:8983_solr c:ducts
>> > s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1>,
>> > collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
>> > - error
>> >
>> > java.net.SocketException: Broken pipe (Write failed)
>> >
>> >        at java.net.SocketOutputStream.socketWrite0(Native Method)
>> > ~[?:1.8.0_232]
>> >
>> >        at
>> > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
>> > ~[?:1.8.0_232]
>> >
>> >        at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
>> > ~[?:1.8.0_232]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159] -
>> > {collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
>> > -
>> > null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: Broken pipe (Write failed)
>> >
>> >       at
>> > org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
>> >
>> >        at
>> > org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
>> >
>> >
>> >
>> >
>> > Thanks
>> >
>> > Fiz..

sample.txt (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Fwd: 2 Async exceptions during distributed update

Fiz Ahmed
Thanks Jorn on getting back to me. Please find attached memory/cpu utilization and complete update process chain.
Please let me know your thoughts.

On Sat, Nov 16, 2019 at 9:09 AM Jörn Franke <[hidden email]> wrote:
Can you please provide the whole update chain ? Below is the directoryFactory and not the update processor chain.

> Am 15.11.2019 um 17:34 schrieb Fiz N <[hidden email]>:
>
> 
> Thanks for your response.
>
> <directoryFactory name="DirectoryFactory"class="${solr.directoryFactory:solr.NRTCachingDirectoryFactory}"/>

> attaching the screenshot of physical memory and cpu.
> Please let me know your thoughts on the below issue.
>
>
>> On Fri, Nov 15, 2019 at 2:18 AM Jörn Franke <[hidden email]> wrote:
>> Do you use a updateprocess factory? How does it look like?
>>
>> What is the physical memory size and CPU?
>> What do you mean by “there are 64 cores sending concurrently?” An application has 64 threats that send concurrently those updates?
>>
>> > Am 15.11.2019 um 02:14 schrieb Fiz N <[hidden email]>:
>> >
>> > Hi Solr Experts,
>> >
>> > SOLR Version : 6.6.2
>> >
>> > OS – Linux 3.1.2
>> >
>> > JDK – 1.8
>> >
>> >
>> >
>> > Shard – 16 – All are active.
>> >
>> > Xms – 16 gb
>> >
>> > Xmx – 16 gb
>> >
>> >
>> >
>> > Schema fields count – 91
>> >
>> > Dynamic fields – 83.
>> >
>> >
>> >
>> > There are multiple threads sending products to solr. tested with batch size
>> > per thread 50 & 100, it worked fine without error and if the batch size is
>> > 1000 the following error occurs.
>> >
>> > there are 64 cores that are sending batches concurrently.
>> >
>> >
>> >
>> > *I am getting the following error when the batch size is 1000. Please
>> > advise.*
>> >
>> >
>> >
>> > 2019-11-14T17:45:44,646 - ERROR [qtp876213901-1169:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard15_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node30, shard=s:shard15}
>> > -
>> > org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > 2 Async exceptions during distributed update:
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> > 10.YYY.40.62:8983 failed to respond
>> >
>> >
>> >
>> > 2019-11-14T17:45:48,821 - ERROR [qtp876213901-202:SolrException@YYY] -
>> > {collection=c:ducts, core=x:ducts_shard7_replica1,
>> > node_name=n:10.YYY.40.81:8983_solr, replica=r:core_node29, shard=s:shard7}
>> > -
>> > org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: 10.YYY.40.81:8983 failed to
>> > respond
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:11,599 - ERROR
>> > [updateExecutor-2-thread-176-processing-x:ducts_shard3_replica2
>> > http:////10.YYY.40.68:8983//solr//ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2> r:core_node26
>> > n:10.YYY.40.68:8983_solr c:ducts
>> > s:shard3:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.68:8983/solr/ducts_shard11_replica2
>> > <http://10.148.40.68:8983/solr/products_shard11_replica2>,
>> > collection=c:ducts, core=x:ducts_shard3_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node26, shard=s:shard3}
>> > - error
>> >
>> > org.apache.http.NoHttpResponseException: 10.YYY.40.68:8983 failed to respond
>> >
>> >        at
>> > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> >        at
>> > org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
>> > ~[httpclient-4.4.1.jar:4.4.1]
>> >
>> >        at
>> > org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
>> > ~[httpcore-4.4.1.jar:4.4.1]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:14,567 - ERROR
>> > [updateExecutor-2-thread-189-processing-x:ducts_shard11_replica2
>> > http:////10.YYY.40.62:8983//solr//ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1> r:core_node25
>> > n:10.YYY.40.68:8983_solr c:ducts
>> > s:shard11:StreamingSolrClients$ErrorReportingConcurrentUpdateSolrClient@131]
>> > - {ConcurrentUpdateSolrClient.url=
>> > http://10.YYY.40.62:8983/solr/ducts_shard2_replica1
>> > <http://10.148.40.62:8983/solr/products_shard2_replica1>,
>> > collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
>> > - error
>> >
>> > java.net.SocketException: Broken pipe (Write failed)
>> >
>> >        at java.net.SocketOutputStream.socketWrite0(Native Method)
>> > ~[?:1.8.0_232]
>> >
>> >        at
>> > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
>> > ~[?:1.8.0_232]
>> >
>> >        at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
>> > ~[?:1.8.0_232]
>> >
>> >
>> >
>> >
>> >
>> > 2019-11-14T19:36:38,851 - ERROR [qtp876213901-542:SolrException@159] -
>> > {collection=c:ducts, core=x:ducts_shard11_replica2,
>> > node_name=n:10.YYY.40.68:8983_solr, replica=r:core_node25, shard=s:shard11}
>> > -
>> > null:org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
>> > Async exception during distributed update: Broken pipe (Write failed)
>> >
>> >       at
>> > org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)
>> >
>> >        at
>> > org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)
>> >
>> >
>> >
>> >
>> > Thanks
>> >
>> > Fiz..

sample.txt (7K) Download Attachment