Delete documents from the Solr index using SolrJ

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Delete documents from the Solr index using SolrJ

Khare, Kushal (MIND)
Hello mates!
I want to know how we can delete the documents from the Solr index . Suppose for my system, I have a document that has been indexed, now its newer version is into use, so I want to use the latest one, for that I want the previous one to be deleted from the index.
Kindly help me a way out !
I went through many articles and blogs, got the way (methods) for deleting , but not actually, how to do it, because it's not possible to delete every time by passing id's in around 50,000 doc system.
Please suggest!

________________________________

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus/trojan/worms/malicious code transmitted by this email. www.motherson.com
Reply | Threaded
Open this post in threaded view
|

Re: Delete documents from the Solr index using SolrJ

Jörn Franke
I don’t understand why it is not possible.

However why don’t you simply overwrite the existing document instead of add+delete

> Am 04.11.2019 um 15:12 schrieb Khare, Kushal (MIND) <[hidden email]>:
>
> Hello mates!
> I want to know how we can delete the documents from the Solr index . Suppose for my system, I have a document that has been indexed, now its newer version is into use, so I want to use the latest one, for that I want the previous one to be deleted from the index.
> Kindly help me a way out !
> I went through many articles and blogs, got the way (methods) for deleting , but not actually, how to do it, because it's not possible to delete every time by passing id's in around 50,000 doc system.
> Please suggest!
>
> ________________________________
>
> The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus/trojan/worms/malicious code transmitted by this email. www.motherson.com
Reply | Threaded
Open this post in threaded view
|

RE: Delete documents from the Solr index using SolrJ

Khare, Kushal (MIND)
Could you please let me know how to achieve that ?


-----Original Message-----
From: Jörn Franke [mailto:[hidden email]]
Sent: 04 November 2019 19:59
To: [hidden email]
Subject: Re: Delete documents from the Solr index using SolrJ

I don’t understand why it is not possible.

However why don’t you simply overwrite the existing document instead of add+delete

> Am 04.11.2019 um 15:12 schrieb Khare, Kushal (MIND) <[hidden email]>:
>
> Hello mates!
> I want to know how we can delete the documents from the Solr index . Suppose for my system, I have a document that has been indexed, now its newer version is into use, so I want to use the latest one, for that I want the previous one to be deleted from the index.
> Kindly help me a way out !
> I went through many articles and blogs, got the way (methods) for deleting , but not actually, how to do it, because it's not possible to delete every time by passing id's in around 50,000 doc system.
> Please suggest!
>
> ________________________________
>
> The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus/trojan/worms/malicious code transmitted by this email. www.motherson.com

________________________________

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus/trojan/worms/malicious code transmitted by this email. www.motherson.com
Reply | Threaded
Open this post in threaded view
|

Re: Delete documents from the Solr index using SolrJ

David Hastings
when you add a new document using the same "id" value as another it just
over writes it

On Mon, Nov 4, 2019 at 9:30 AM Khare, Kushal (MIND) <
[hidden email]> wrote:

> Could you please let me know how to achieve that ?
>
>
> -----Original Message-----
> From: Jörn Franke [mailto:[hidden email]]
> Sent: 04 November 2019 19:59
> To: [hidden email]
> Subject: Re: Delete documents from the Solr index using SolrJ
>
> I don’t understand why it is not possible.
>
> However why don’t you simply overwrite the existing document instead of
> add+delete
>
> > Am 04.11.2019 um 15:12 schrieb Khare, Kushal (MIND) <
> [hidden email]>:
> >
> > Hello mates!
> > I want to know how we can delete the documents from the Solr index .
> Suppose for my system, I have a document that has been indexed, now its
> newer version is into use, so I want to use the latest one, for that I want
> the previous one to be deleted from the index.
> > Kindly help me a way out !
> > I went through many articles and blogs, got the way (methods) for
> deleting , but not actually, how to do it, because it's not possible to
> delete every time by passing id's in around 50,000 doc system.
> > Please suggest!
> >
> > ________________________________
> >
> > The information contained in this electronic message and any attachments
> to this message are intended for the exclusive use of the addressee(s) and
> may contain proprietary, confidential or privileged information. If you are
> not the intended recipient, you should not disseminate, distribute or copy
> this e-mail. Please notify the sender immediately and destroy all copies of
> this message and any attachments. WARNING: Computer viruses can be
> transmitted via email. The recipient should check this email and any
> attachments for the presence of viruses. The company accepts no liability
> for any damage caused by any virus/trojan/worms/malicious code transmitted
> by this email. www.motherson.com
>
> ________________________________
>
> The information contained in this electronic message and any attachments
> to this message are intended for the exclusive use of the addressee(s) and
> may contain proprietary, confidential or privileged information. If you are
> not the intended recipient, you should not disseminate, distribute or copy
> this e-mail. Please notify the sender immediately and destroy all copies of
> this message and any attachments. WARNING: Computer viruses can be
> transmitted via email. The recipient should check this email and any
> attachments for the presence of viruses. The company accepts no liability
> for any damage caused by any virus/trojan/worms/malicious code transmitted
> by this email. www.motherson.com
>
Reply | Threaded
Open this post in threaded view
|

RE: Delete documents from the Solr index using SolrJ

Khare, Kushal (MIND)
In reply to this post by Khare, Kushal (MIND)
Basically , what I need is to refresh the index. Suppose, in a directory I have 4 docs, that have been indexed. So, my search works upon those 4.
Now, when I delete one of them, re-index and search, still that deleted document from the directory is being searched upon.
Hope I have made it a bit more clear now.

-----Original Message-----
From: Khare, Kushal (MIND) [mailto:[hidden email]]
Sent: 04 November 2019 20:00
To: [hidden email]
Subject: RE: Delete documents from the Solr index using SolrJ

Could you please let me know how to achieve that ?


-----Original Message-----
From: Jörn Franke [mailto:[hidden email]]
Sent: 04 November 2019 19:59
To: [hidden email]
Subject: Re: Delete documents from the Solr index using SolrJ

I don’t understand why it is not possible.

However why don’t you simply overwrite the existing document instead of add+delete

> Am 04.11.2019 um 15:12 schrieb Khare, Kushal (MIND) <[hidden email]>:
>
> Hello mates!
> I want to know how we can delete the documents from the Solr index . Suppose for my system, I have a document that has been indexed, now its newer version is into use, so I want to use the latest one, for that I want the previous one to be deleted from the index.
> Kindly help me a way out !
> I went through many articles and blogs, got the way (methods) for deleting , but not actually, how to do it, because it's not possible to delete every time by passing id's in around 50,000 doc system.
> Please suggest!
>
> ________________________________
>
> The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus/trojan/worms/malicious code transmitted by this email. www.motherson.com

________________________________

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus/trojan/worms/malicious code transmitted by this email. www.motherson.com

________________________________

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus/trojan/worms/malicious code transmitted by this email. www.motherson.com
Reply | Threaded
Open this post in threaded view
|

RE: Delete documents from the Solr index using SolrJ

Khare, Kushal (MIND)
In reply to this post by David Hastings
In my case, id won't be same.
Suppose, I have a doc with id : 20
Now, it's newer version would be either 20.1 or 22
What in this case?
-----Original Message-----
From: David Hastings [mailto:[hidden email]]
Sent: 04 November 2019 20:04
To: [hidden email]
Subject: Re: Delete documents from the Solr index using SolrJ

when you add a new document using the same "id" value as another it just over writes it

On Mon, Nov 4, 2019 at 9:30 AM Khare, Kushal (MIND) < [hidden email]> wrote:

> Could you please let me know how to achieve that ?
>
>
> -----Original Message-----
> From: Jörn Franke [mailto:[hidden email]]
> Sent: 04 November 2019 19:59
> To: [hidden email]
> Subject: Re: Delete documents from the Solr index using SolrJ
>
> I don’t understand why it is not possible.
>
> However why don’t you simply overwrite the existing document instead
> of
> add+delete
>
> > Am 04.11.2019 um 15:12 schrieb Khare, Kushal (MIND) <
> [hidden email]>:
> >
> > Hello mates!
> > I want to know how we can delete the documents from the Solr index .
> Suppose for my system, I have a document that has been indexed, now
> its newer version is into use, so I want to use the latest one, for
> that I want the previous one to be deleted from the index.
> > Kindly help me a way out !
> > I went through many articles and blogs, got the way (methods) for
> deleting , but not actually, how to do it, because it's not possible
> to delete every time by passing id's in around 50,000 doc system.
> > Please suggest!
> >
> > ________________________________
> >
> > The information contained in this electronic message and any
> > attachments
> to this message are intended for the exclusive use of the addressee(s)
> and may contain proprietary, confidential or privileged information.
> If you are not the intended recipient, you should not disseminate,
> distribute or copy this e-mail. Please notify the sender immediately
> and destroy all copies of this message and any attachments. WARNING:
> Computer viruses can be transmitted via email. The recipient should
> check this email and any attachments for the presence of viruses. The
> company accepts no liability for any damage caused by any
> virus/trojan/worms/malicious code transmitted by this email.
> www.motherson.com
>
> ________________________________
>
> The information contained in this electronic message and any
> attachments to this message are intended for the exclusive use of the
> addressee(s) and may contain proprietary, confidential or privileged
> information. If you are not the intended recipient, you should not
> disseminate, distribute or copy this e-mail. Please notify the sender
> immediately and destroy all copies of this message and any
> attachments. WARNING: Computer viruses can be transmitted via email.
> The recipient should check this email and any attachments for the
> presence of viruses. The company accepts no liability for any damage
> caused by any virus/trojan/worms/malicious code transmitted by this
> email. www.motherson.com
>

________________________________

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus/trojan/worms/malicious code transmitted by this email. www.motherson.com
Reply | Threaded
Open this post in threaded view
|

Re: Delete documents from the Solr index using SolrJ

David Hastings
delete them by query would do the trick unless im missing something
significant in what youre trying to do here. you can just pass in an xml
command:
'<delete><query>".$kill_query."</query></delete>'

On Mon, Nov 4, 2019 at 9:37 AM Khare, Kushal (MIND) <
[hidden email]> wrote:

> In my case, id won't be same.
> Suppose, I have a doc with id : 20
> Now, it's newer version would be either 20.1 or 22
> What in this case?
> -----Original Message-----
> From: David Hastings [mailto:[hidden email]]
> Sent: 04 November 2019 20:04
> To: [hidden email]
> Subject: Re: Delete documents from the Solr index using SolrJ
>
> when you add a new document using the same "id" value as another it just
> over writes it
>
> On Mon, Nov 4, 2019 at 9:30 AM Khare, Kushal (MIND) <
> [hidden email]> wrote:
>
> > Could you please let me know how to achieve that ?
> >
> >
> > -----Original Message-----
> > From: Jörn Franke [mailto:[hidden email]]
> > Sent: 04 November 2019 19:59
> > To: [hidden email]
> > Subject: Re: Delete documents from the Solr index using SolrJ
> >
> > I don’t understand why it is not possible.
> >
> > However why don’t you simply overwrite the existing document instead
> > of
> > add+delete
> >
> > > Am 04.11.2019 um 15:12 schrieb Khare, Kushal (MIND) <
> > [hidden email]>:
> > >
> > > Hello mates!
> > > I want to know how we can delete the documents from the Solr index .
> > Suppose for my system, I have a document that has been indexed, now
> > its newer version is into use, so I want to use the latest one, for
> > that I want the previous one to be deleted from the index.
> > > Kindly help me a way out !
> > > I went through many articles and blogs, got the way (methods) for
> > deleting , but not actually, how to do it, because it's not possible
> > to delete every time by passing id's in around 50,000 doc system.
> > > Please suggest!
> > >
> > > ________________________________
> > >
> > > The information contained in this electronic message and any
> > > attachments
> > to this message are intended for the exclusive use of the addressee(s)
> > and may contain proprietary, confidential or privileged information.
> > If you are not the intended recipient, you should not disseminate,
> > distribute or copy this e-mail. Please notify the sender immediately
> > and destroy all copies of this message and any attachments. WARNING:
> > Computer viruses can be transmitted via email. The recipient should
> > check this email and any attachments for the presence of viruses. The
> > company accepts no liability for any damage caused by any
> > virus/trojan/worms/malicious code transmitted by this email.
> > www.motherson.com
> >
> > ________________________________
> >
> > The information contained in this electronic message and any
> > attachments to this message are intended for the exclusive use of the
> > addressee(s) and may contain proprietary, confidential or privileged
> > information. If you are not the intended recipient, you should not
> > disseminate, distribute or copy this e-mail. Please notify the sender
> > immediately and destroy all copies of this message and any
> > attachments. WARNING: Computer viruses can be transmitted via email.
> > The recipient should check this email and any attachments for the
> > presence of viruses. The company accepts no liability for any damage
> > caused by any virus/trojan/worms/malicious code transmitted by this
> > email. www.motherson.com
> >
>
> ________________________________
>
> The information contained in this electronic message and any attachments
> to this message are intended for the exclusive use of the addressee(s) and
> may contain proprietary, confidential or privileged information. If you are
> not the intended recipient, you should not disseminate, distribute or copy
> this e-mail. Please notify the sender immediately and destroy all copies of
> this message and any attachments. WARNING: Computer viruses can be
> transmitted via email. The recipient should check this email and any
> attachments for the presence of viruses. The company accepts no liability
> for any damage caused by any virus/trojan/worms/malicious code transmitted
> by this email. www.motherson.com
>
Reply | Threaded
Open this post in threaded view
|

RE: Delete documents from the Solr index using SolrJ

Khare, Kushal (MIND)
Thanks!
Actually am working on a Java web application using SolrJ for Solr search.
The users would actually be uploading/editing/deleting the docs. What have done is defined a location/directory where the docs would be stored and passed that location for indexing.
So, I am quite confused how to carry on with the solution that you proposed. Please guide !

-----Original Message-----
From: David Hastings [mailto:[hidden email]]
Sent: 04 November 2019 20:10
To: [hidden email]
Subject: Re: Delete documents from the Solr index using SolrJ

delete them by query would do the trick unless im missing something significant in what youre trying to do here. you can just pass in an xml
command:
'<delete><query>".$kill_query."</query></delete>'

On Mon, Nov 4, 2019 at 9:37 AM Khare, Kushal (MIND) < [hidden email]> wrote:

> In my case, id won't be same.
> Suppose, I have a doc with id : 20
> Now, it's newer version would be either 20.1 or 22 What in this case?
> -----Original Message-----
> From: David Hastings [mailto:[hidden email]]
> Sent: 04 November 2019 20:04
> To: [hidden email]
> Subject: Re: Delete documents from the Solr index using SolrJ
>
> when you add a new document using the same "id" value as another it
> just over writes it
>
> On Mon, Nov 4, 2019 at 9:30 AM Khare, Kushal (MIND) <
> [hidden email]> wrote:
>
> > Could you please let me know how to achieve that ?
> >
> >
> > -----Original Message-----
> > From: Jörn Franke [mailto:[hidden email]]
> > Sent: 04 November 2019 19:59
> > To: [hidden email]
> > Subject: Re: Delete documents from the Solr index using SolrJ
> >
> > I don’t understand why it is not possible.
> >
> > However why don’t you simply overwrite the existing document instead
> > of
> > add+delete
> >
> > > Am 04.11.2019 um 15:12 schrieb Khare, Kushal (MIND) <
> > [hidden email]>:
> > >
> > > Hello mates!
> > > I want to know how we can delete the documents from the Solr index .
> > Suppose for my system, I have a document that has been indexed, now
> > its newer version is into use, so I want to use the latest one, for
> > that I want the previous one to be deleted from the index.
> > > Kindly help me a way out !
> > > I went through many articles and blogs, got the way (methods) for
> > deleting , but not actually, how to do it, because it's not possible
> > to delete every time by passing id's in around 50,000 doc system.
> > > Please suggest!
> > >
> > > ________________________________
> > >
> > > The information contained in this electronic message and any
> > > attachments
> > to this message are intended for the exclusive use of the
> > addressee(s) and may contain proprietary, confidential or privileged information.
> > If you are not the intended recipient, you should not disseminate,
> > distribute or copy this e-mail. Please notify the sender immediately
> > and destroy all copies of this message and any attachments. WARNING:
> > Computer viruses can be transmitted via email. The recipient should
> > check this email and any attachments for the presence of viruses.
> > The company accepts no liability for any damage caused by any
> > virus/trojan/worms/malicious code transmitted by this email.
> > www.motherson.com
> >
> > ________________________________
> >
> > The information contained in this electronic message and any
> > attachments to this message are intended for the exclusive use of
> > the
> > addressee(s) and may contain proprietary, confidential or privileged
> > information. If you are not the intended recipient, you should not
> > disseminate, distribute or copy this e-mail. Please notify the
> > sender immediately and destroy all copies of this message and any
> > attachments. WARNING: Computer viruses can be transmitted via email.
> > The recipient should check this email and any attachments for the
> > presence of viruses. The company accepts no liability for any damage
> > caused by any virus/trojan/worms/malicious code transmitted by this
> > email. www.motherson.com
> >
>
> ________________________________
>
> The information contained in this electronic message and any
> attachments to this message are intended for the exclusive use of the
> addressee(s) and may contain proprietary, confidential or privileged
> information. If you are not the intended recipient, you should not
> disseminate, distribute or copy this e-mail. Please notify the sender
> immediately and destroy all copies of this message and any
> attachments. WARNING: Computer viruses can be transmitted via email.
> The recipient should check this email and any attachments for the
> presence of viruses. The company accepts no liability for any damage
> caused by any virus/trojan/worms/malicious code transmitted by this
> email. www.motherson.com
>

________________________________

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus/trojan/worms/malicious code transmitted by this email. www.motherson.com
Reply | Threaded
Open this post in threaded view
|

RE: Delete documents from the Solr index using SolrJ

lancasp22
You can delete documents in SolrJ by using deleteByQuery. Using this you can delete any number of documents from your index or all your documents depending on the query you specify as the parameter. How you use it is down to your application.

You haven't said if your application performs a full re-index, but if so you might find it useful to index a version number for your data which you increment each time you perform the full indexing. Then you can increment version, re-index data, delete data for old version number.


-----Original Message-----
From: Khare, Kushal (MIND) [mailto:[hidden email]]
Sent: 04 November 2019 15:03
To: [hidden email]
Subject: [EXTERNAL] RE: Delete documents from the Solr index using SolrJ

Thanks!
Actually am working on a Java web application using SolrJ for Solr search.
The users would actually be uploading/editing/deleting the docs. What have done is defined a location/directory where the docs would be stored and passed that location for indexing.
So, I am quite confused how to carry on with the solution that you proposed. Please guide !

-----Original Message-----
From: David Hastings [mailto:[hidden email]]
Sent: 04 November 2019 20:10
To: [hidden email]
Subject: Re: Delete documents from the Solr index using SolrJ

delete them by query would do the trick unless im missing something significant in what youre trying to do here. you can just pass in an xml
command:
'<delete><query>".$kill_query."</query></delete>'

On Mon, Nov 4, 2019 at 9:37 AM Khare, Kushal (MIND) < [hidden email]> wrote:

> In my case, id won't be same.
> Suppose, I have a doc with id : 20
> Now, it's newer version would be either 20.1 or 22 What in this case?
> -----Original Message-----
> From: David Hastings [mailto:[hidden email]]
> Sent: 04 November 2019 20:04
> To: [hidden email]
> Subject: Re: Delete documents from the Solr index using SolrJ
>
> when you add a new document using the same "id" value as another it
> just over writes it
>
> On Mon, Nov 4, 2019 at 9:30 AM Khare, Kushal (MIND) <
> [hidden email]> wrote:
>
> > Could you please let me know how to achieve that ?
> >
> >
> > -----Original Message-----
> > From: Jörn Franke [mailto:[hidden email]]
> > Sent: 04 November 2019 19:59
> > To: [hidden email]
> > Subject: Re: Delete documents from the Solr index using SolrJ
> >
> > I don’t understand why it is not possible.
> >
> > However why don’t you simply overwrite the existing document instead
> > of
> > add+delete
> >
> > > Am 04.11.2019 um 15:12 schrieb Khare, Kushal (MIND) <
> > [hidden email]>:
> > >
> > > Hello mates!
> > > I want to know how we can delete the documents from the Solr index .
> > Suppose for my system, I have a document that has been indexed, now
> > its newer version is into use, so I want to use the latest one, for
> > that I want the previous one to be deleted from the index.
> > > Kindly help me a way out !
> > > I went through many articles and blogs, got the way (methods) for
> > deleting , but not actually, how to do it, because it's not possible
> > to delete every time by passing id's in around 50,000 doc system.
> > > Please suggest!
> > >
> > > ________________________________
> > >
> > > The information contained in this electronic message and any
> > > attachments
> > to this message are intended for the exclusive use of the
> > addressee(s) and may contain proprietary, confidential or privileged information.
> > If you are not the intended recipient, you should not disseminate,
> > distribute or copy this e-mail. Please notify the sender immediately
> > and destroy all copies of this message and any attachments. WARNING:
> > Computer viruses can be transmitted via email. The recipient should
> > check this email and any attachments for the presence of viruses.
> > The company accepts no liability for any damage caused by any
> > virus/trojan/worms/malicious code transmitted by this email.
> > www.motherson.com
> >
> > ________________________________
> >
> > The information contained in this electronic message and any
> > attachments to this message are intended for the exclusive use of
> > the
> > addressee(s) and may contain proprietary, confidential or privileged
> > information. If you are not the intended recipient, you should not
> > disseminate, distribute or copy this e-mail. Please notify the
> > sender immediately and destroy all copies of this message and any
> > attachments. WARNING: Computer viruses can be transmitted via email.
> > The recipient should check this email and any attachments for the
> > presence of viruses. The company accepts no liability for any damage
> > caused by any virus/trojan/worms/malicious code transmitted by this
> > email. www.motherson.com
> >
>
> ________________________________
>
> The information contained in this electronic message and any
> attachments to this message are intended for the exclusive use of the
> addressee(s) and may contain proprietary, confidential or privileged
> information. If you are not the intended recipient, you should not
> disseminate, distribute or copy this e-mail. Please notify the sender
> immediately and destroy all copies of this message and any
> attachments. WARNING: Computer viruses can be transmitted via email.
> The recipient should check this email and any attachments for the
> presence of viruses. The company accepts no liability for any damage
> caused by any virus/trojan/worms/malicious code transmitted by this
> email. www.motherson.com
>

________________________________

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus/trojan/worms/malicious code transmitted by this email. www.motherson.com
________________________________

This message is confidential and may contain privileged information. You should not disclose its contents to any other person. If you are not the intended recipient, please notify the sender named above immediately. It is expressly declared that this e-mail does not constitute nor form part of a contract or unilateral obligation. Opinions, conclusions and other information in this message that do not relate to the official business of findmypast shall be understood as neither given nor endorsed by it.
________________________________
Reply | Threaded
Open this post in threaded view
|

Re: Delete documents from the Solr index using SolrJ

Walter Underwood
In reply to this post by Khare, Kushal (MIND)
If it is the same document, why are you changing the ID? Use the same ID and you are done. You won’t need to delete previous versions.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)

> On Nov 4, 2019, at 8:37 AM, Khare, Kushal (MIND) <[hidden email]> wrote:
>
> In my case, id won't be same.
> Suppose, I have a doc with id : 20
> Now, it's newer version would be either 20.1 or 22
> What in this case?
> -----Original Message-----
> From: David Hastings [mailto:[hidden email]]
> Sent: 04 November 2019 20:04
> To: [hidden email]
> Subject: Re: Delete documents from the Solr index using SolrJ
>
> when you add a new document using the same "id" value as another it just over writes it
>
> On Mon, Nov 4, 2019 at 9:30 AM Khare, Kushal (MIND) < [hidden email]> wrote:
>
>> Could you please let me know how to achieve that ?
>>
>>
>> -----Original Message-----
>> From: Jörn Franke [mailto:[hidden email]]
>> Sent: 04 November 2019 19:59
>> To: [hidden email]
>> Subject: Re: Delete documents from the Solr index using SolrJ
>>
>> I don’t understand why it is not possible.
>>
>> However why don’t you simply overwrite the existing document instead
>> of
>> add+delete
>>
>>> Am 04.11.2019 um 15:12 schrieb Khare, Kushal (MIND) <
>> [hidden email]>:
>>>
>>> Hello mates!
>>> I want to know how we can delete the documents from the Solr index .
>> Suppose for my system, I have a document that has been indexed, now
>> its newer version is into use, so I want to use the latest one, for
>> that I want the previous one to be deleted from the index.
>>> Kindly help me a way out !
>>> I went through many articles and blogs, got the way (methods) for
>> deleting , but not actually, how to do it, because it's not possible
>> to delete every time by passing id's in around 50,000 doc system.
>>> Please suggest!
>>>
>>> ________________________________
>>>
>>> The information contained in this electronic message and any
>>> attachments
>> to this message are intended for the exclusive use of the addressee(s)
>> and may contain proprietary, confidential or privileged information.
>> If you are not the intended recipient, you should not disseminate,
>> distribute or copy this e-mail. Please notify the sender immediately
>> and destroy all copies of this message and any attachments. WARNING:
>> Computer viruses can be transmitted via email. The recipient should
>> check this email and any attachments for the presence of viruses. The
>> company accepts no liability for any damage caused by any
>> virus/trojan/worms/malicious code transmitted by this email.
>> www.motherson.com
>>
>> ________________________________
>>
>> The information contained in this electronic message and any
>> attachments to this message are intended for the exclusive use of the
>> addressee(s) and may contain proprietary, confidential or privileged
>> information. If you are not the intended recipient, you should not
>> disseminate, distribute or copy this e-mail. Please notify the sender
>> immediately and destroy all copies of this message and any
>> attachments. WARNING: Computer viruses can be transmitted via email.
>> The recipient should check this email and any attachments for the
>> presence of viruses. The company accepts no liability for any damage
>> caused by any virus/trojan/worms/malicious code transmitted by this
>> email. www.motherson.com
>>
>
> ________________________________
>
> The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus/trojan/worms/malicious code transmitted by this email. www.motherson.com

Reply | Threaded
Open this post in threaded view
|

Re: Delete documents from the Solr index using SolrJ

Erick Erickson
What Walter said. If you require displaying the version number in the UI, put that in a separate field.

BTW, Delete-by-query can be expensive for various arcane reasons if you’re using SolrCloud.

> On Nov 4, 2019, at 11:08 AM, Walter Underwood <[hidden email]> wrote:
>
> If it is the same document, why are you changing the ID? Use the same ID and you are done. You won’t need to delete previous versions.
>
> wunder
> Walter Underwood
> [hidden email]
> http://observer.wunderwood.org/  (my blog)
>
>> On Nov 4, 2019, at 8:37 AM, Khare, Kushal (MIND) <[hidden email]> wrote:
>>
>> In my case, id won't be same.
>> Suppose, I have a doc with id : 20
>> Now, it's newer version would be either 20.1 or 22
>> What in this case?
>> -----Original Message-----
>> From: David Hastings [mailto:[hidden email]]
>> Sent: 04 November 2019 20:04
>> To: [hidden email]
>> Subject: Re: Delete documents from the Solr index using SolrJ
>>
>> when you add a new document using the same "id" value as another it just over writes it
>>
>> On Mon, Nov 4, 2019 at 9:30 AM Khare, Kushal (MIND) < [hidden email]> wrote:
>>
>>> Could you please let me know how to achieve that ?
>>>
>>>
>>> -----Original Message-----
>>> From: Jörn Franke [mailto:[hidden email]]
>>> Sent: 04 November 2019 19:59
>>> To: [hidden email]
>>> Subject: Re: Delete documents from the Solr index using SolrJ
>>>
>>> I don’t understand why it is not possible.
>>>
>>> However why don’t you simply overwrite the existing document instead
>>> of
>>> add+delete
>>>
>>>> Am 04.11.2019 um 15:12 schrieb Khare, Kushal (MIND) <
>>> [hidden email]>:
>>>>
>>>> Hello mates!
>>>> I want to know how we can delete the documents from the Solr index .
>>> Suppose for my system, I have a document that has been indexed, now
>>> its newer version is into use, so I want to use the latest one, for
>>> that I want the previous one to be deleted from the index.
>>>> Kindly help me a way out !
>>>> I went through many articles and blogs, got the way (methods) for
>>> deleting , but not actually, how to do it, because it's not possible
>>> to delete every time by passing id's in around 50,000 doc system.
>>>> Please suggest!
>>>>
>>>> ________________________________
>>>>
>>>> The information contained in this electronic message and any
>>>> attachments
>>> to this message are intended for the exclusive use of the addressee(s)
>>> and may contain proprietary, confidential or privileged information.
>>> If you are not the intended recipient, you should not disseminate,
>>> distribute or copy this e-mail. Please notify the sender immediately
>>> and destroy all copies of this message and any attachments. WARNING:
>>> Computer viruses can be transmitted via email. The recipient should
>>> check this email and any attachments for the presence of viruses. The
>>> company accepts no liability for any damage caused by any
>>> virus/trojan/worms/malicious code transmitted by this email.
>>> www.motherson.com
>>>
>>> ________________________________
>>>
>>> The information contained in this electronic message and any
>>> attachments to this message are intended for the exclusive use of the
>>> addressee(s) and may contain proprietary, confidential or privileged
>>> information. If you are not the intended recipient, you should not
>>> disseminate, distribute or copy this e-mail. Please notify the sender
>>> immediately and destroy all copies of this message and any
>>> attachments. WARNING: Computer viruses can be transmitted via email.
>>> The recipient should check this email and any attachments for the
>>> presence of viruses. The company accepts no liability for any damage
>>> caused by any virus/trojan/worms/malicious code transmitted by this
>>> email. www.motherson.com
>>>
>>
>> ________________________________
>>
>> The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus/trojan/worms/malicious code transmitted by this email. www.motherson.com
>

Reply | Threaded
Open this post in threaded view
|

RE: Delete documents from the Solr index using SolrJ

Khare, Kushal (MIND)
In reply to this post by lancasp22
Well, I cannot still completely relate to the solutions by you guys, am looking into it as how could I achieve that with my application. Thanks !
One thing, that I want to know is how to avoid full re-indexing, that is, what I need is I don’t want that Solr index all the data every time some docs are added, instead I want it to update it, that is index only newly added docs. I hope this is possible, but how ?
Because, currently I am using SolrJ  and it re-index complete data each time.

-----Original Message-----
From: Peter Lancaster [mailto:[hidden email]]
Sent: 04 November 2019 21:35
To: [hidden email]
Subject: RE: Delete documents from the Solr index using SolrJ

You can delete documents in SolrJ by using deleteByQuery. Using this you can delete any number of documents from your index or all your documents depending on the query you specify as the parameter. How you use it is down to your application.

You haven't said if your application performs a full re-index, but if so you might find it useful to index a version number for your data which you increment each time you perform the full indexing. Then you can increment version, re-index data, delete data for old version number.


-----Original Message-----
From: Khare, Kushal (MIND) [mailto:[hidden email]]
Sent: 04 November 2019 15:03
To: [hidden email]
Subject: [EXTERNAL] RE: Delete documents from the Solr index using SolrJ

Thanks!
Actually am working on a Java web application using SolrJ for Solr search.
The users would actually be uploading/editing/deleting the docs. What have done is defined a location/directory where the docs would be stored and passed that location for indexing.
So, I am quite confused how to carry on with the solution that you proposed. Please guide !

-----Original Message-----
From: David Hastings [mailto:[hidden email]]
Sent: 04 November 2019 20:10
To: [hidden email]
Subject: Re: Delete documents from the Solr index using SolrJ

delete them by query would do the trick unless im missing something significant in what youre trying to do here. you can just pass in an xml
command:
'<delete><query>".$kill_query."</query></delete>'

On Mon, Nov 4, 2019 at 9:37 AM Khare, Kushal (MIND) < [hidden email]> wrote:

> In my case, id won't be same.
> Suppose, I have a doc with id : 20
> Now, it's newer version would be either 20.1 or 22 What in this case?
> -----Original Message-----
> From: David Hastings [mailto:[hidden email]]
> Sent: 04 November 2019 20:04
> To: [hidden email]
> Subject: Re: Delete documents from the Solr index using SolrJ
>
> when you add a new document using the same "id" value as another it
> just over writes it
>
> On Mon, Nov 4, 2019 at 9:30 AM Khare, Kushal (MIND) <
> [hidden email]> wrote:
>
> > Could you please let me know how to achieve that ?
> >
> >
> > -----Original Message-----
> > From: Jörn Franke [mailto:[hidden email]]
> > Sent: 04 November 2019 19:59
> > To: [hidden email]
> > Subject: Re: Delete documents from the Solr index using SolrJ
> >
> > I don’t understand why it is not possible.
> >
> > However why don’t you simply overwrite the existing document instead
> > of
> > add+delete
> >
> > > Am 04.11.2019 um 15:12 schrieb Khare, Kushal (MIND) <
> > [hidden email]>:
> > >
> > > Hello mates!
> > > I want to know how we can delete the documents from the Solr index .
> > Suppose for my system, I have a document that has been indexed, now
> > its newer version is into use, so I want to use the latest one, for
> > that I want the previous one to be deleted from the index.
> > > Kindly help me a way out !
> > > I went through many articles and blogs, got the way (methods) for
> > deleting , but not actually, how to do it, because it's not possible
> > to delete every time by passing id's in around 50,000 doc system.
> > > Please suggest!
> > >
> > > ________________________________
> > >
> > > The information contained in this electronic message and any
> > > attachments
> > to this message are intended for the exclusive use of the
> > addressee(s) and may contain proprietary, confidential or privileged information.
> > If you are not the intended recipient, you should not disseminate,
> > distribute or copy this e-mail. Please notify the sender immediately
> > and destroy all copies of this message and any attachments. WARNING:
> > Computer viruses can be transmitted via email. The recipient should
> > check this email and any attachments for the presence of viruses.
> > The company accepts no liability for any damage caused by any
> > virus/trojan/worms/malicious code transmitted by this email.
> > www.motherson.com
> >
> > ________________________________
> >
> > The information contained in this electronic message and any
> > attachments to this message are intended for the exclusive use of
> > the
> > addressee(s) and may contain proprietary, confidential or privileged
> > information. If you are not the intended recipient, you should not
> > disseminate, distribute or copy this e-mail. Please notify the
> > sender immediately and destroy all copies of this message and any
> > attachments. WARNING: Computer viruses can be transmitted via email.
> > The recipient should check this email and any attachments for the
> > presence of viruses. The company accepts no liability for any damage
> > caused by any virus/trojan/worms/malicious code transmitted by this
> > email. www.motherson.com
> >
>
> ________________________________
>
> The information contained in this electronic message and any
> attachments to this message are intended for the exclusive use of the
> addressee(s) and may contain proprietary, confidential or privileged
> information. If you are not the intended recipient, you should not
> disseminate, distribute or copy this e-mail. Please notify the sender
> immediately and destroy all copies of this message and any
> attachments. WARNING: Computer viruses can be transmitted via email.
> The recipient should check this email and any attachments for the
> presence of viruses. The company accepts no liability for any damage
> caused by any virus/trojan/worms/malicious code transmitted by this
> email. www.motherson.com
>

________________________________

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus/trojan/worms/malicious code transmitted by this email. www.motherson.com ________________________________

This message is confidential and may contain privileged information. You should not disclose its contents to any other person. If you are not the intended recipient, please notify the sender named above immediately. It is expressly declared that this e-mail does not constitute nor form part of a contract or unilateral obligation. Opinions, conclusions and other information in this message that do not relate to the official business of findmypast shall be understood as neither given nor endorsed by it.
________________________________

________________________________

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus/trojan/worms/malicious code transmitted by this email. www.motherson.com
Reply | Threaded
Open this post in threaded view
|

Re: Delete documents from the Solr index using SolrJ

Erick Erickson
OK, you have two options:

1.1> do NOT construct IDs with the version. Have two separate fields, id (which is the <uniqueKey> in your schema and a _separate_ field called tracking (note, there’s already by default an _version_ field, with underscores, for optimistic locking, do not use that).

1.2> Index the new version of the doc with the exact same ID and a new version and a new “tracking” value

Solr will replace the old version with the new version based on the ID.

Second:
Before you re-add the doc, issue a delete-by-query that identifies the document, something like q=id:123*

_How_ you determine that there is a new version of the doc you need to index is outside of Solr, you have to do that yourself.

Best,
Erick

> On Nov 5, 2019, at 3:56 AM, Khare, Kushal (MIND) <[hidden email]> wrote:
>
> Well, I cannot still completely relate to the solutions by you guys, am looking into it as how could I achieve that with my application. Thanks !
> One thing, that I want to know is how to avoid full re-indexing, that is, what I need is I don’t want that Solr index all the data every time some docs are added, instead I want it to update it, that is index only newly added docs. I hope this is possible, but how ?
> Because, currently I am using SolrJ  and it re-index complete data each time.
>
> -----Original Message-----
> From: Peter Lancaster [mailto:[hidden email]]
> Sent: 04 November 2019 21:35
> To: [hidden email]
> Subject: RE: Delete documents from the Solr index using SolrJ
>
> You can delete documents in SolrJ by using deleteByQuery. Using this you can delete any number of documents from your index or all your documents depending on the query you specify as the parameter. How you use it is down to your application.
>
> You haven't said if your application performs a full re-index, but if so you might find it useful to index a version number for your data which you increment each time you perform the full indexing. Then you can increment version, re-index data, delete data for old version number.
>
>
> -----Original Message-----
> From: Khare, Kushal (MIND) [mailto:[hidden email]]
> Sent: 04 November 2019 15:03
> To: [hidden email]
> Subject: [EXTERNAL] RE: Delete documents from the Solr index using SolrJ
>
> Thanks!
> Actually am working on a Java web application using SolrJ for Solr search.
> The users would actually be uploading/editing/deleting the docs. What have done is defined a location/directory where the docs would be stored and passed that location for indexing.
> So, I am quite confused how to carry on with the solution that you proposed. Please guide !
>
> -----Original Message-----
> From: David Hastings [mailto:[hidden email]]
> Sent: 04 November 2019 20:10
> To: [hidden email]
> Subject: Re: Delete documents from the Solr index using SolrJ
>
> delete them by query would do the trick unless im missing something significant in what youre trying to do here. you can just pass in an xml
> command:
> '<delete><query>".$kill_query."</query></delete>'
>
> On Mon, Nov 4, 2019 at 9:37 AM Khare, Kushal (MIND) < [hidden email]> wrote:
>
>> In my case, id won't be same.
>> Suppose, I have a doc with id : 20
>> Now, it's newer version would be either 20.1 or 22 What in this case?
>> -----Original Message-----
>> From: David Hastings [mailto:[hidden email]]
>> Sent: 04 November 2019 20:04
>> To: [hidden email]
>> Subject: Re: Delete documents from the Solr index using SolrJ
>>
>> when you add a new document using the same "id" value as another it
>> just over writes it
>>
>> On Mon, Nov 4, 2019 at 9:30 AM Khare, Kushal (MIND) <
>> [hidden email]> wrote:
>>
>>> Could you please let me know how to achieve that ?
>>>
>>>
>>> -----Original Message-----
>>> From: Jörn Franke [mailto:[hidden email]]
>>> Sent: 04 November 2019 19:59
>>> To: [hidden email]
>>> Subject: Re: Delete documents from the Solr index using SolrJ
>>>
>>> I don’t understand why it is not possible.
>>>
>>> However why don’t you simply overwrite the existing document instead
>>> of
>>> add+delete
>>>
>>>> Am 04.11.2019 um 15:12 schrieb Khare, Kushal (MIND) <
>>> [hidden email]>:
>>>>
>>>> Hello mates!
>>>> I want to know how we can delete the documents from the Solr index .
>>> Suppose for my system, I have a document that has been indexed, now
>>> its newer version is into use, so I want to use the latest one, for
>>> that I want the previous one to be deleted from the index.
>>>> Kindly help me a way out !
>>>> I went through many articles and blogs, got the way (methods) for
>>> deleting , but not actually, how to do it, because it's not possible
>>> to delete every time by passing id's in around 50,000 doc system.
>>>> Please suggest!
>>>>
>>>> ________________________________
>>>>
>>>> The information contained in this electronic message and any
>>>> attachments
>>> to this message are intended for the exclusive use of the
>>> addressee(s) and may contain proprietary, confidential or privileged information.
>>> If you are not the intended recipient, you should not disseminate,
>>> distribute or copy this e-mail. Please notify the sender immediately
>>> and destroy all copies of this message and any attachments. WARNING:
>>> Computer viruses can be transmitted via email. The recipient should
>>> check this email and any attachments for the presence of viruses.
>>> The company accepts no liability for any damage caused by any
>>> virus/trojan/worms/malicious code transmitted by this email.
>>> www.motherson.com
>>>
>>> ________________________________
>>>
>>> The information contained in this electronic message and any
>>> attachments to this message are intended for the exclusive use of
>>> the
>>> addressee(s) and may contain proprietary, confidential or privileged
>>> information. If you are not the intended recipient, you should not
>>> disseminate, distribute or copy this e-mail. Please notify the
>>> sender immediately and destroy all copies of this message and any
>>> attachments. WARNING: Computer viruses can be transmitted via email.
>>> The recipient should check this email and any attachments for the
>>> presence of viruses. The company accepts no liability for any damage
>>> caused by any virus/trojan/worms/malicious code transmitted by this
>>> email. www.motherson.com
>>>
>>
>> ________________________________
>>
>> The information contained in this electronic message and any
>> attachments to this message are intended for the exclusive use of the
>> addressee(s) and may contain proprietary, confidential or privileged
>> information. If you are not the intended recipient, you should not
>> disseminate, distribute or copy this e-mail. Please notify the sender
>> immediately and destroy all copies of this message and any
>> attachments. WARNING: Computer viruses can be transmitted via email.
>> The recipient should check this email and any attachments for the
>> presence of viruses. The company accepts no liability for any damage
>> caused by any virus/trojan/worms/malicious code transmitted by this
>> email. www.motherson.com
>>
>
> ________________________________
>
> The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus/trojan/worms/malicious code transmitted by this email. www.motherson.com ________________________________
>
> This message is confidential and may contain privileged information. You should not disclose its contents to any other person. If you are not the intended recipient, please notify the sender named above immediately. It is expressly declared that this e-mail does not constitute nor form part of a contract or unilateral obligation. Opinions, conclusions and other information in this message that do not relate to the official business of findmypast shall be understood as neither given nor endorsed by it.
> ________________________________
>
> ________________________________
>
> The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus/trojan/worms/malicious code transmitted by this email. www.motherson.com