Prevent Solr overwriting documents

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Prevent Solr overwriting documents

Aaron Hoffer
We want to prevent Solr from overwriting an existing document if document's
ID already exists in the core.

This unit test fails because the update/overwrite is permitted:

public void testUpdateProhibited() {
  final Index index = baseInstance();
  indexRepository.save(index);
  Index index0 = indexRepository.findById(INDEX_ID).get();
  index0.setContents("AAA");
  indexRepository.save(index0);
  Index index1 = indexRepository.findById(INDEX_ID).get();
  assertThat(index, equalTo(index1));
}


The failure is:
Expected: <Index(contents=AAA, id=a2d8464ed41b4766a888617ecbcac573, ...>
but: was <Index(contents=Winterfell, id=a2d8464ed41b4766a888617ecbcac573,
...>

What do I need to do prevent the second save from overwriting the existing
document?

I commented out the updateHandler in the solr config file, to no effect.
We are using Spring Data with Solr 8.1.
In the core's schema, id is defined as unique  like this:
<uniqueKey>id</uniqueKey>
Reply | Threaded
Open this post in threaded view
|

Re: Prevent Solr overwriting documents

Alexandre Rafalovitch
How about Optimistic Concurrency with _version_ set to negative value?

You could inject that extra value in URP chain if need be.

Regards,
    Alex

On Wed, Nov 27, 2019, 5:41 PM Aaron Hoffer, <[hidden email]> wrote:

> We want to prevent Solr from overwriting an existing document if document's
> ID already exists in the core.
>
> This unit test fails because the update/overwrite is permitted:
>
> public void testUpdateProhibited() {
>   final Index index = baseInstance();
>   indexRepository.save(index);
>   Index index0 = indexRepository.findById(INDEX_ID).get();
>   index0.setContents("AAA");
>   indexRepository.save(index0);
>   Index index1 = indexRepository.findById(INDEX_ID).get();
>   assertThat(index, equalTo(index1));
> }
>
>
> The failure is:
> Expected: <Index(contents=AAA, id=a2d8464ed41b4766a888617ecbcac573, ...>
> but: was <Index(contents=Winterfell, id=a2d8464ed41b4766a888617ecbcac573,
> ...>
>
> What do I need to do prevent the second save from overwriting the existing
> document?
>
> I commented out the updateHandler in the solr config file, to no effect.
> We are using Spring Data with Solr 8.1.
> In the core's schema, id is defined as unique  like this:
> <uniqueKey>id</uniqueKey>
>
Reply | Threaded
Open this post in threaded view
|

Re: Prevent Solr overwriting documents

Alexandre Rafalovitch
Oops. And the link...
https://lucene.apache.org/solr/guide/6_6/updating-parts-of-documents.html#UpdatingPartsofDocuments-OptimisticConcurrency

On Wed, Nov 27, 2019, 6:24 PM Alexandre Rafalovitch, <[hidden email]>
wrote:

> How about Optimistic Concurrency with _version_ set to negative value?
>
> You could inject that extra value in URP chain if need be.
>
> Regards,
>     Alex
>
> On Wed, Nov 27, 2019, 5:41 PM Aaron Hoffer, <[hidden email]> wrote:
>
>> We want to prevent Solr from overwriting an existing document if
>> document's
>> ID already exists in the core.
>>
>> This unit test fails because the update/overwrite is permitted:
>>
>> public void testUpdateProhibited() {
>>   final Index index = baseInstance();
>>   indexRepository.save(index);
>>   Index index0 = indexRepository.findById(INDEX_ID).get();
>>   index0.setContents("AAA");
>>   indexRepository.save(index0);
>>   Index index1 = indexRepository.findById(INDEX_ID).get();
>>   assertThat(index, equalTo(index1));
>> }
>>
>>
>> The failure is:
>> Expected: <Index(contents=AAA, id=a2d8464ed41b4766a888617ecbcac573, ...>
>> but: was <Index(contents=Winterfell, id=a2d8464ed41b4766a888617ecbcac573,
>> ...>
>>
>> What do I need to do prevent the second save from overwriting the existing
>> document?
>>
>> I commented out the updateHandler in the solr config file, to no effect.
>> We are using Spring Data with Solr 8.1.
>> In the core's schema, id is defined as unique  like this:
>> <uniqueKey>id</uniqueKey>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Prevent Solr overwriting documents

Walter Underwood
In reply to this post by Alexandre Rafalovitch
Even if that works, it is evil as something to leave in a client codebase. Maybe a do-no-overwrite flag would be useful.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)

> On Nov 27, 2019, at 3:24 PM, Alexandre Rafalovitch <[hidden email]> wrote:
>
> How about Optimistic Concurrency with _version_ set to negative value?
>
> You could inject that extra value in URP chain if need be.
>
> Regards,
>    Alex
>
> On Wed, Nov 27, 2019, 5:41 PM Aaron Hoffer, <[hidden email]> wrote:
>
>> We want to prevent Solr from overwriting an existing document if document's
>> ID already exists in the core.
>>
>> This unit test fails because the update/overwrite is permitted:
>>
>> public void testUpdateProhibited() {
>>  final Index index = baseInstance();
>>  indexRepository.save(index);
>>  Index index0 = indexRepository.findById(INDEX_ID).get();
>>  index0.setContents("AAA");
>>  indexRepository.save(index0);
>>  Index index1 = indexRepository.findById(INDEX_ID).get();
>>  assertThat(index, equalTo(index1));
>> }
>>
>>
>> The failure is:
>> Expected: <Index(contents=AAA, id=a2d8464ed41b4766a888617ecbcac573, ...>
>> but: was <Index(contents=Winterfell, id=a2d8464ed41b4766a888617ecbcac573,
>> ...>
>>
>> What do I need to do prevent the second save from overwriting the existing
>> document?
>>
>> I commented out the updateHandler in the solr config file, to no effect.
>> We are using Spring Data with Solr 8.1.
>> In the core's schema, id is defined as unique  like this:
>> <uniqueKey>id</uniqueKey>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Prevent Solr overwriting documents

Walter Underwood
That would be “do-not-overwrite”.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)

> On Nov 27, 2019, at 4:38 PM, Walter Underwood <[hidden email]> wrote:
>
> Even if that works, it is evil as something to leave in a client codebase. Maybe a do-no-overwrite flag would be useful.
>
> wunder
> Walter Underwood
> [hidden email]
> http://observer.wunderwood.org/  (my blog)
>
>> On Nov 27, 2019, at 3:24 PM, Alexandre Rafalovitch <[hidden email]> wrote:
>>
>> How about Optimistic Concurrency with _version_ set to negative value?
>>
>> You could inject that extra value in URP chain if need be.
>>
>> Regards,
>>   Alex
>>
>> On Wed, Nov 27, 2019, 5:41 PM Aaron Hoffer, <[hidden email]> wrote:
>>
>>> We want to prevent Solr from overwriting an existing document if document's
>>> ID already exists in the core.
>>>
>>> This unit test fails because the update/overwrite is permitted:
>>>
>>> public void testUpdateProhibited() {
>>> final Index index = baseInstance();
>>> indexRepository.save(index);
>>> Index index0 = indexRepository.findById(INDEX_ID).get();
>>> index0.setContents("AAA");
>>> indexRepository.save(index0);
>>> Index index1 = indexRepository.findById(INDEX_ID).get();
>>> assertThat(index, equalTo(index1));
>>> }
>>>
>>>
>>> The failure is:
>>> Expected: <Index(contents=AAA, id=a2d8464ed41b4766a888617ecbcac573, ...>
>>> but: was <Index(contents=Winterfell, id=a2d8464ed41b4766a888617ecbcac573,
>>> ...>
>>>
>>> What do I need to do prevent the second save from overwriting the existing
>>> document?
>>>
>>> I commented out the updateHandler in the solr config file, to no effect.
>>> We are using Spring Data with Solr 8.1.
>>> In the core's schema, id is defined as unique  like this:
>>> <uniqueKey>id</uniqueKey>
>>>
>