duplicate doc of uniqueKey

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

duplicate doc of uniqueKey

Novin Novin
Hi Guys,

I end up with duplicate docs in solr cloud. I don't know how to debug it.
So looking for help here please.

Below is details:
Solr 6.6.2
zookeeper 3.4.10

Below is example of duplicate record of Json:

{
  "responseHeader":{
    "zkConnected":true,
    "status":0,
    "QTime":0,
    "params":{
      "q":"*:*",
      "distrib":"false",
      "indent":"on",
      "fl":"id",
      "fq":"id:mid531281",
      "wt":"json"}},
  "response":{"numFound":2,"start":0,"docs":[
      {
        "id":"mid531281"},
      {
        "id":"mid531281"}]
  }}

schema file contains:
<field name="id" type="string" indexed="true" stored="true" required="true"
multiValued="false" docValues="true"/>

<uniqueKey>id</uniqueKey>

Let me know if extra information required. Any help would be really
appreciated.

Regards,
Novin
Reply | Threaded
Open this post in threaded view
|

Re: duplicate doc of uniqueKey

Erick Erickson
Also ask for the _version_ field in your fl list. The _version_ field
is used o r optimistic locking. This is mostly a curiosity
question....

The only time I've ever seen something like this is if you, for
instance, use MergeIndexes or MapReduceIndexerTool (which does a
MergeIndexes under the covers). Have you done anything similar?

Best,
Erick


On Thu, Apr 19, 2018 at 8:54 AM, Novin Novin <[hidden email]> wrote:

> Hi Guys,
>
> I end up with duplicate docs in solr cloud. I don't know how to debug it.
> So looking for help here please.
>
> Below is details:
> Solr 6.6.2
> zookeeper 3.4.10
>
> Below is example of duplicate record of Json:
>
> {
>   "responseHeader":{
>     "zkConnected":true,
>     "status":0,
>     "QTime":0,
>     "params":{
>       "q":"*:*",
>       "distrib":"false",
>       "indent":"on",
>       "fl":"id",
>       "fq":"id:mid531281",
>       "wt":"json"}},
>   "response":{"numFound":2,"start":0,"docs":[
>       {
>         "id":"mid531281"},
>       {
>         "id":"mid531281"}]
>   }}
>
> schema file contains:
> <field name="id" type="string" indexed="true" stored="true" required="true"
> multiValued="false" docValues="true"/>
>
> <uniqueKey>id</uniqueKey>
>
> Let me know if extra information required. Any help would be really
> appreciated.
>
> Regards,
> Novin
Reply | Threaded
Open this post in threaded view
|

Re: duplicate doc of uniqueKey

Novin Novin
Hi Erick,

I haven't done any of merge indexes with MergeIndexes or
MapReduceIndexerTool.
Actually I found that one of doc does not have child doc, because I am
using solr parent child docs for block join queries. As far as I know, it
is know issue for parent child docs that if you send only parent doc it end
up as single doc rather than to replace parent with child.
If you know that this issue has been fixed with certain solr version please
let me know or any other way to handle this issue.

Thanks in advance,
Novin



On Thu, 19 Apr 2018 at 17:26 Erick Erickson <[hidden email]> wrote:

> Also ask for the _version_ field in your fl list. The _version_ field
> is used o r optimistic locking. This is mostly a curiosity
> question....
>
> The only time I've ever seen something like this is if you, for
> instance, use MergeIndexes or MapReduceIndexerTool (which does a
> MergeIndexes under the covers). Have you done anything similar?
>
> Best,
> Erick
>
>
> On Thu, Apr 19, 2018 at 8:54 AM, Novin Novin <[hidden email]> wrote:
> > Hi Guys,
> >
> > I end up with duplicate docs in solr cloud. I don't know how to debug it.
> > So looking for help here please.
> >
> > Below is details:
> > Solr 6.6.2
> > zookeeper 3.4.10
> >
> > Below is example of duplicate record of Json:
> >
> > {
> >   "responseHeader":{
> >     "zkConnected":true,
> >     "status":0,
> >     "QTime":0,
> >     "params":{
> >       "q":"*:*",
> >       "distrib":"false",
> >       "indent":"on",
> >       "fl":"id",
> >       "fq":"id:mid531281",
> >       "wt":"json"}},
> >   "response":{"numFound":2,"start":0,"docs":[
> >       {
> >         "id":"mid531281"},
> >       {
> >         "id":"mid531281"}]
> >   }}
> >
> > schema file contains:
> > <field name="id" type="string" indexed="true" stored="true"
> required="true"
> > multiValued="false" docValues="true"/>
> >
> > <uniqueKey>id</uniqueKey>
> >
> > Let me know if extra information required. Any help would be really
> > appreciated.
> >
> > Regards,
> > Novin
>
Reply | Threaded
Open this post in threaded view
|

Re: duplicate doc of uniqueKey

Karthik Ramachandran
Novin,

Was your system time moved to future time and then was reset to current
time?

Solr will add the new document and will send delete for the old document
but there will no document matching the criteria.


On Thu, Apr 19, 2018 at 1:10 PM, Novin Novin <[hidden email]> wrote:

> Hi Erick,
>
> I haven't done any of merge indexes with MergeIndexes or
> MapReduceIndexerTool.
> Actually I found that one of doc does not have child doc, because I am
> using solr parent child docs for block join queries. As far as I know, it
> is know issue for parent child docs that if you send only parent doc it end
> up as single doc rather than to replace parent with child.
> If you know that this issue has been fixed with certain solr version please
> let me know or any other way to handle this issue.
>
> Thanks in advance,
> Novin
>
>
>
> On Thu, 19 Apr 2018 at 17:26 Erick Erickson <[hidden email]>
> wrote:
>
> > Also ask for the _version_ field in your fl list. The _version_ field
> > is used o r optimistic locking. This is mostly a curiosity
> > question....
> >
> > The only time I've ever seen something like this is if you, for
> > instance, use MergeIndexes or MapReduceIndexerTool (which does a
> > MergeIndexes under the covers). Have you done anything similar?
> >
> > Best,
> > Erick
> >
> >
> > On Thu, Apr 19, 2018 at 8:54 AM, Novin Novin <[hidden email]>
> wrote:
> > > Hi Guys,
> > >
> > > I end up with duplicate docs in solr cloud. I don't know how to debug
> it.
> > > So looking for help here please.
> > >
> > > Below is details:
> > > Solr 6.6.2
> > > zookeeper 3.4.10
> > >
> > > Below is example of duplicate record of Json:
> > >
> > > {
> > >   "responseHeader":{
> > >     "zkConnected":true,
> > >     "status":0,
> > >     "QTime":0,
> > >     "params":{
> > >       "q":"*:*",
> > >       "distrib":"false",
> > >       "indent":"on",
> > >       "fl":"id",
> > >       "fq":"id:mid531281",
> > >       "wt":"json"}},
> > >   "response":{"numFound":2,"start":0,"docs":[
> > >       {
> > >         "id":"mid531281"},
> > >       {
> > >         "id":"mid531281"}]
> > >   }}
> > >
> > > schema file contains:
> > > <field name="id" type="string" indexed="true" stored="true"
> > required="true"
> > > multiValued="false" docValues="true"/>
> > >
> > > <uniqueKey>id</uniqueKey>
> > >
> > > Let me know if extra information required. Any help would be really
> > > appreciated.
> > >
> > > Regards,
> > > Novin
> >
>



--
With Thanks & Regards
Karthik Ramachandran

P Please don't print this e-mail unless you really need to
Reply | Threaded
Open this post in threaded view
|

Re: duplicate doc of uniqueKey

Novin Novin
Hi Karthik,

*Was your system time moved to future time and then was reset to current*
*time?*

Nothing happen like this as far as I known.

Thanks in advance
Novin


On Thu, 19 Apr 2018 at 18:26 Karthik Ramachandran <[hidden email]> wrote:

> Novin,
>
> Was your system time moved to future time and then was reset to current
> time?
>
> Solr will add the new document and will send delete for the old document
> but there will no document matching the criteria.
>
>
> On Thu, Apr 19, 2018 at 1:10 PM, Novin Novin <[hidden email]> wrote:
>
> > Hi Erick,
> >
> > I haven't done any of merge indexes with MergeIndexes or
> > MapReduceIndexerTool.
> > Actually I found that one of doc does not have child doc, because I am
> > using solr parent child docs for block join queries. As far as I know, it
> > is know issue for parent child docs that if you send only parent doc it
> end
> > up as single doc rather than to replace parent with child.
> > If you know that this issue has been fixed with certain solr version
> please
> > let me know or any other way to handle this issue.
> >
> > Thanks in advance,
> > Novin
> >
> >
> >
> > On Thu, 19 Apr 2018 at 17:26 Erick Erickson <[hidden email]>
> > wrote:
> >
> > > Also ask for the _version_ field in your fl list. The _version_ field
> > > is used o r optimistic locking. This is mostly a curiosity
> > > question....
> > >
> > > The only time I've ever seen something like this is if you, for
> > > instance, use MergeIndexes or MapReduceIndexerTool (which does a
> > > MergeIndexes under the covers). Have you done anything similar?
> > >
> > > Best,
> > > Erick
> > >
> > >
> > > On Thu, Apr 19, 2018 at 8:54 AM, Novin Novin <[hidden email]>
> > wrote:
> > > > Hi Guys,
> > > >
> > > > I end up with duplicate docs in solr cloud. I don't know how to debug
> > it.
> > > > So looking for help here please.
> > > >
> > > > Below is details:
> > > > Solr 6.6.2
> > > > zookeeper 3.4.10
> > > >
> > > > Below is example of duplicate record of Json:
> > > >
> > > > {
> > > >   "responseHeader":{
> > > >     "zkConnected":true,
> > > >     "status":0,
> > > >     "QTime":0,
> > > >     "params":{
> > > >       "q":"*:*",
> > > >       "distrib":"false",
> > > >       "indent":"on",
> > > >       "fl":"id",
> > > >       "fq":"id:mid531281",
> > > >       "wt":"json"}},
> > > >   "response":{"numFound":2,"start":0,"docs":[
> > > >       {
> > > >         "id":"mid531281"},
> > > >       {
> > > >         "id":"mid531281"}]
> > > >   }}
> > > >
> > > > schema file contains:
> > > > <field name="id" type="string" indexed="true" stored="true"
> > > required="true"
> > > > multiValued="false" docValues="true"/>
> > > >
> > > > <uniqueKey>id</uniqueKey>
> > > >
> > > > Let me know if extra information required. Any help would be really
> > > > appreciated.
> > > >
> > > > Regards,
> > > > Novin
> > >
> >
>
>
>
> --
> With Thanks & Regards
> Karthik Ramachandran
>
> P Please don't print this e-mail unless you really need to
>
Reply | Threaded
Open this post in threaded view
|

Re: duplicate doc of uniqueKey

Erick Erickson
Right, parent/child docs _must_ be treated as a block. By that I mean
you cannot add/delete individuals child docs and/or parent docs.
That's one of the limitations of parent/child blocks and I don't know
of any plans to change that.

Best,
Erick

On Thu, Apr 19, 2018 at 11:14 AM, Novin Novin <[hidden email]> wrote:

> Hi Karthik,
>
> *Was your system time moved to future time and then was reset to current*
> *time?*
>
> Nothing happen like this as far as I known.
>
> Thanks in advance
> Novin
>
>
> On Thu, 19 Apr 2018 at 18:26 Karthik Ramachandran <[hidden email]> wrote:
>
>> Novin,
>>
>> Was your system time moved to future time and then was reset to current
>> time?
>>
>> Solr will add the new document and will send delete for the old document
>> but there will no document matching the criteria.
>>
>>
>> On Thu, Apr 19, 2018 at 1:10 PM, Novin Novin <[hidden email]> wrote:
>>
>> > Hi Erick,
>> >
>> > I haven't done any of merge indexes with MergeIndexes or
>> > MapReduceIndexerTool.
>> > Actually I found that one of doc does not have child doc, because I am
>> > using solr parent child docs for block join queries. As far as I know, it
>> > is know issue for parent child docs that if you send only parent doc it
>> end
>> > up as single doc rather than to replace parent with child.
>> > If you know that this issue has been fixed with certain solr version
>> please
>> > let me know or any other way to handle this issue.
>> >
>> > Thanks in advance,
>> > Novin
>> >
>> >
>> >
>> > On Thu, 19 Apr 2018 at 17:26 Erick Erickson <[hidden email]>
>> > wrote:
>> >
>> > > Also ask for the _version_ field in your fl list. The _version_ field
>> > > is used o r optimistic locking. This is mostly a curiosity
>> > > question....
>> > >
>> > > The only time I've ever seen something like this is if you, for
>> > > instance, use MergeIndexes or MapReduceIndexerTool (which does a
>> > > MergeIndexes under the covers). Have you done anything similar?
>> > >
>> > > Best,
>> > > Erick
>> > >
>> > >
>> > > On Thu, Apr 19, 2018 at 8:54 AM, Novin Novin <[hidden email]>
>> > wrote:
>> > > > Hi Guys,
>> > > >
>> > > > I end up with duplicate docs in solr cloud. I don't know how to debug
>> > it.
>> > > > So looking for help here please.
>> > > >
>> > > > Below is details:
>> > > > Solr 6.6.2
>> > > > zookeeper 3.4.10
>> > > >
>> > > > Below is example of duplicate record of Json:
>> > > >
>> > > > {
>> > > >   "responseHeader":{
>> > > >     "zkConnected":true,
>> > > >     "status":0,
>> > > >     "QTime":0,
>> > > >     "params":{
>> > > >       "q":"*:*",
>> > > >       "distrib":"false",
>> > > >       "indent":"on",
>> > > >       "fl":"id",
>> > > >       "fq":"id:mid531281",
>> > > >       "wt":"json"}},
>> > > >   "response":{"numFound":2,"start":0,"docs":[
>> > > >       {
>> > > >         "id":"mid531281"},
>> > > >       {
>> > > >         "id":"mid531281"}]
>> > > >   }}
>> > > >
>> > > > schema file contains:
>> > > > <field name="id" type="string" indexed="true" stored="true"
>> > > required="true"
>> > > > multiValued="false" docValues="true"/>
>> > > >
>> > > > <uniqueKey>id</uniqueKey>
>> > > >
>> > > > Let me know if extra information required. Any help would be really
>> > > > appreciated.
>> > > >
>> > > > Regards,
>> > > > Novin
>> > >
>> >
>>
>>
>>
>> --
>> With Thanks & Regards
>> Karthik Ramachandran
>>
>> P Please don't print this e-mail unless you really need to
>>
Reply | Threaded
Open this post in threaded view
|

Re: duplicate doc of uniqueKey

Novin Novin
Thanks Erick and Karthik for you help.

On Thu, 19 Apr 2018 at 19:53 Erick Erickson <[hidden email]> wrote:

> Right, parent/child docs _must_ be treated as a block. By that I mean
> you cannot add/delete individuals child docs and/or parent docs.
> That's one of the limitations of parent/child blocks and I don't know
> of any plans to change that.
>
> Best,
> Erick
>
> On Thu, Apr 19, 2018 at 11:14 AM, Novin Novin <[hidden email]> wrote:
> > Hi Karthik,
> >
> > *Was your system time moved to future time and then was reset to current*
> > *time?*
> >
> > Nothing happen like this as far as I known.
> >
> > Thanks in advance
> > Novin
> >
> >
> > On Thu, 19 Apr 2018 at 18:26 Karthik Ramachandran <[hidden email]>
> wrote:
> >
> >> Novin,
> >>
> >> Was your system time moved to future time and then was reset to current
> >> time?
> >>
> >> Solr will add the new document and will send delete for the old document
> >> but there will no document matching the criteria.
> >>
> >>
> >> On Thu, Apr 19, 2018 at 1:10 PM, Novin Novin <[hidden email]>
> wrote:
> >>
> >> > Hi Erick,
> >> >
> >> > I haven't done any of merge indexes with MergeIndexes or
> >> > MapReduceIndexerTool.
> >> > Actually I found that one of doc does not have child doc, because I am
> >> > using solr parent child docs for block join queries. As far as I
> know, it
> >> > is know issue for parent child docs that if you send only parent doc
> it
> >> end
> >> > up as single doc rather than to replace parent with child.
> >> > If you know that this issue has been fixed with certain solr version
> >> please
> >> > let me know or any other way to handle this issue.
> >> >
> >> > Thanks in advance,
> >> > Novin
> >> >
> >> >
> >> >
> >> > On Thu, 19 Apr 2018 at 17:26 Erick Erickson <[hidden email]>
> >> > wrote:
> >> >
> >> > > Also ask for the _version_ field in your fl list. The _version_
> field
> >> > > is used o r optimistic locking. This is mostly a curiosity
> >> > > question....
> >> > >
> >> > > The only time I've ever seen something like this is if you, for
> >> > > instance, use MergeIndexes or MapReduceIndexerTool (which does a
> >> > > MergeIndexes under the covers). Have you done anything similar?
> >> > >
> >> > > Best,
> >> > > Erick
> >> > >
> >> > >
> >> > > On Thu, Apr 19, 2018 at 8:54 AM, Novin Novin <[hidden email]>
> >> > wrote:
> >> > > > Hi Guys,
> >> > > >
> >> > > > I end up with duplicate docs in solr cloud. I don't know how to
> debug
> >> > it.
> >> > > > So looking for help here please.
> >> > > >
> >> > > > Below is details:
> >> > > > Solr 6.6.2
> >> > > > zookeeper 3.4.10
> >> > > >
> >> > > > Below is example of duplicate record of Json:
> >> > > >
> >> > > > {
> >> > > >   "responseHeader":{
> >> > > >     "zkConnected":true,
> >> > > >     "status":0,
> >> > > >     "QTime":0,
> >> > > >     "params":{
> >> > > >       "q":"*:*",
> >> > > >       "distrib":"false",
> >> > > >       "indent":"on",
> >> > > >       "fl":"id",
> >> > > >       "fq":"id:mid531281",
> >> > > >       "wt":"json"}},
> >> > > >   "response":{"numFound":2,"start":0,"docs":[
> >> > > >       {
> >> > > >         "id":"mid531281"},
> >> > > >       {
> >> > > >         "id":"mid531281"}]
> >> > > >   }}
> >> > > >
> >> > > > schema file contains:
> >> > > > <field name="id" type="string" indexed="true" stored="true"
> >> > > required="true"
> >> > > > multiValued="false" docValues="true"/>
> >> > > >
> >> > > > <uniqueKey>id</uniqueKey>
> >> > > >
> >> > > > Let me know if extra information required. Any help would be
> really
> >> > > > appreciated.
> >> > > >
> >> > > > Regards,
> >> > > > Novin
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> With Thanks & Regards
> >> Karthik Ramachandran
> >>
> >> P Please don't print this e-mail unless you really need to
> >>
>