Issue while using Document Routing in SolrCloud 6.1

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Issue while using Document Routing in SolrCloud 6.1

Ketan Thanki
Hi,

Need the help regarding to below mention query.
I have configure the 2 collections with each has 2 shard and 2 replica and i have implemented Composite document routing for my unique field 'Id' where I have use 2 level Tenant route as mentions below.
e.g : projectId:158380 modelId:3606 where tenants bits use as projectId/2!modelId/8 for below ID
"id":"79190!450!0003606#001#001#0#002754269#11760499351"

Issue: its seems that my retrieval get faster but the insertion was slower compared to without routing changes

Please  do needful.

Regards,
Ketan.

Please cast a vote for Asite in the 2017 Construction Computing Awards: Click here to Vote<http://caddealer.com/concompawards/index.php?page=cca2017vote>

[CC Award Winners!]

Reply | Threaded
Open this post in threaded view
|

Re: Issue while using Document Routing in SolrCloud 6.1

Emir Arnautović
Hi Ketan,
Is it possible that you are indexing only one tenant and that is causing single shard to become hotspot?

Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 10 Oct 2017, at 12:47, Ketan Thanki <[hidden email]> wrote:
>
> Hi,
>
> Need the help regarding to below mention query.
> I have configure the 2 collections with each has 2 shard and 2 replica and i have implemented Composite document routing for my unique field 'Id' where I have use 2 level Tenant route as mentions below.
> e.g : projectId:158380 modelId:3606 where tenants bits use as projectId/2!modelId/8 for below ID
> "id":"79190!450!0003606#001#001#0#002754269#11760499351"
>
> Issue: its seems that my retrieval get faster but the insertion was slower compared to without routing changes
>
> Please  do needful.
>
> Regards,
> Ketan.
>
> Please cast a vote for Asite in the 2017 Construction Computing Awards: Click here to Vote<http://caddealer.com/concompawards/index.php?page=cca2017vote>
>
> [CC Award Winners!]
>

Reply | Threaded
Open this post in threaded view
|

FW: Issue while using Document Routing in SolrCloud 6.1

Ketan Thanki
In reply to this post by Ketan Thanki

Thanks Emir,

As mentions below, I have indexing the using two tenant and my data are currently belongs to only one shard which also shows impact in retrieval much faster but While insert its seems slower.
So is there any specific reason for that.

Please do needful.

Regards,
Ketan.



Hi Ketan,
Is it possible that you are indexing only one tenant and that is causing single shard to become
hotspot?

Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/


From: Ketan Thanki
Sent: Tuesday, October 10, 2017 4:18 PM
To: '[hidden email]'
Subject: Issue while using Document Routing in SolrCloud 6.1

Hi,

Need the help regarding to below mention query.
I have configure the 2 collections with each has 2 shard and 2 replica and i have implemented Composite document routing for my unique field 'Id' where I have use 2 level Tenant route as mentions below.
e.g : projectId:158380 modelId:3606 where tenants bits use as projectId/2!modelId/8 for below ID
"id":"79190!450!0003606#001#001#0#002754269#11760499351"

Issue: its seems that my retrieval get faster but the insertion was slower compared to without routing changes

Please  do needful.

Regards,
Ketan.

Reply | Threaded
Open this post in threaded view
|

Re: Issue while using Document Routing in SolrCloud 6.1

Emir Arnautović
Hi Ketan,
Each shard is a separate index and if you are indexing 100doc/sec without routing with two shards, you are indexing 50 docs/shard. If you have routing, and all documents are from single tenant, single shard has to be able to process 100doc/sec. If you have two nodes it means that you have to process same number of documents with half of resources. But even with one node it is expected to be slower because more docs/sec means more merging and expected to have lower throughput.
If you change your tests to index both tenants at the same time, you should not experience lower throughput.
When it comes to search latency, it is because you do not have merge phase in your query and when you talk about search throughput in addition to latency, you use half of resources so you can expect to double the throughput with routing, in your case.

HTH,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 11 Oct 2017, at 07:31, Ketan Thanki <[hidden email]> wrote:
>
>
> Thanks Emir,
>
> As mentions below, I have indexing the using two tenant and my data are currently belongs to only one shard which also shows impact in retrieval much faster but While insert its seems slower.
> So is there any specific reason for that.
>
> Please do needful.
>
> Regards,
> Ketan.
>
>
>
> Hi Ketan,
> Is it possible that you are indexing only one tenant and that is causing single shard to become
> hotspot?
>
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>
>
> From: Ketan Thanki
> Sent: Tuesday, October 10, 2017 4:18 PM
> To: '[hidden email]'
> Subject: Issue while using Document Routing in SolrCloud 6.1
>
> Hi,
>
> Need the help regarding to below mention query.
> I have configure the 2 collections with each has 2 shard and 2 replica and i have implemented Composite document routing for my unique field 'Id' where I have use 2 level Tenant route as mentions below.
> e.g : projectId:158380 modelId:3606 where tenants bits use as projectId/2!modelId/8 for below ID
> "id":"79190!450!0003606#001#001#0#002754269#11760499351"
>
> Issue: its seems that my retrieval get faster but the insertion was slower compared to without routing changes
>
> Please  do needful.
>
> Regards,
> Ketan.
>