CompositeId router using custom route field for updates and atomic update

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

CompositeId router using custom route field for updates and atomic update

Niko Himanen
Hello,

I came up with a situation with collection created with "router.field" and using atomic update format for route.field in document that documents were routed into wrong shard in CompositeIdRouter. 

After doing some investigation I noticed that CompositeIdRouter#sliceHash takes field value used for routing as is, which means that atomic update format (like set=123) is used as a whole to calculate route hash instead of just value 123.

I came over this by using field for routing which is never atomically updated, but I feel like this is still quite nasty feature/bug which is hard to detect.

Is this a known issue or should I create ticket from it?

Br,

Niko Himanen
Reply | Threaded
Open this post in threaded view
|

Re: CompositeId router using custom route field for updates and atomic update

Andrzej Białecki
Hi Niko,

Please create a Jira issue, this looks like a bug. It also needs more discussion - I’m not convinced we should allow updates (atomic or not) to the id field, because (as the name suggests) this field defines the identity of the document, and if the identity is modified is it still the same document that we should be updating? ;)  

> On 18 Apr 2019, at 12:30, Niko Himanen <[hidden email]> wrote:
>
> Hello,
>
> I came up with a situation with collection created with "router.field" and using atomic update format for route.field in document that documents were routed into wrong shard in CompositeIdRouter.
>
> After doing some investigation I noticed that CompositeIdRouter#sliceHash takes field value used for routing as is, which means that atomic update format (like set=123) is used as a whole to calculate route hash instead of just value 123.
>
> I came over this by using field for routing which is never atomically updated, but I feel like this is still quite nasty feature/bug which is hard to detect.
>
> Is this a known issue or should I create ticket from it?
>
> Br,
>
> Niko Himanen


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: CompositeId router using custom route field for updates and atomic update

Niko Himanen-2
Hey Andrzej,

Thank you for your response.

In this specific case updated field is not id or unique field, it is just some random field I am using for routing. So in this case, it may make sense to be able to route document to new location by updating (atomic or not) :).

I would think that solution is either to use real value for routing or throw exception if field is atomically updated and there is a change that thats why document would end up in wrong shard using current logic.

Anyway. I created ticket for discussion: https://issues.apache.org/jira/browse/SOLR-13411

On Thu, Apr 18, 2019 at 2:26 PM Andrzej Białecki <[hidden email]> wrote:
Hi Niko,

Please create a Jira issue, this looks like a bug. It also needs more discussion - I’m not convinced we should allow updates (atomic or not) to the id field, because (as the name suggests) this field defines the identity of the document, and if the identity is modified is it still the same document that we should be updating? ;) 

> On 18 Apr 2019, at 12:30, Niko Himanen <[hidden email]> wrote:
>
> Hello,
>
> I came up with a situation with collection created with "router.field" and using atomic update format for route.field in document that documents were routed into wrong shard in CompositeIdRouter.
>
> After doing some investigation I noticed that CompositeIdRouter#sliceHash takes field value used for routing as is, which means that atomic update format (like set=123) is used as a whole to calculate route hash instead of just value 123.
>
> I came over this by using field for routing which is never atomically updated, but I feel like this is still quite nasty feature/bug which is hard to detect.
>
> Is this a known issue or should I create ticket from it?
>
> Br,
>
> Niko Himanen


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



--
Niko Himanen
Senior Search Engineer
M: +358 504 100 773

AlphaSense  |  www.alpha-sense.com


AlphaSense-Signature-V3.gif (2K) Download Attachment