initiate reindexing in solr for field type changes

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

initiate reindexing in solr for field type changes

Darniz
Hi users,
i wanted to know is there a way we can initialte solr indexing.
I mean for example i have a field which was of type string and i indexed 100 documents.
When i change the field to text i dont want to load the document again, i should be able to just run a command line and the documents should be indexed again with the updated field type.

The same applies with stemmer.
for example we were using Porter stemmer and we indexed like 2 million docs.
Now we want to change the configuration to Kstemmer and for this to take effect we have to again bulk load all 2 million docs.

Is there a way to run the indexer again.

Please let me know.
Thanks
darniz
Reply | Threaded
Open this post in threaded view
|

Re: initiate reindexing in solr for field type changes

Otis Gospodnetic-2
Darniz,

The "indexer" is typically an external application you write.  This application gets documents from some data source and sends them to Solr for indexing.  It is this application that needs to be able to re-get the appropriate set of documents from the data source and re-send them to Solr for re-indexing.

Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR



----- Original Message ----

> From: darniz <[hidden email]>
> To: [hidden email]
> Sent: Tue, November 17, 2009 5:12:09 PM
> Subject: initiate reindexing in solr for field type changes
>
>
> Hi users,
> i wanted to know is there a way we can initialte solr indexing.
> I mean for example i have a field which was of type string and i indexed 100
> documents.
> When i change the field to text i dont want to load the document again, i
> should be able to just run a command line and the documents should be
> indexed again with the updated field type.
>
> The same applies with stemmer.
> for example we were using Porter stemmer and we indexed like 2 million docs.
> Now we want to change the configuration to Kstemmer and for this to take
> effect we have to again bulk load all 2 million docs.
>
> Is there a way to run the indexer again.
>
> Please let me know.
> Thanks
> darniz
> --
> View this message in context:
> http://old.nabble.com/initiate-reindexing-in-solr-for-field-type-changes-tp26397067p26397067.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: initiate reindexing in solr for field type changes

Darniz
Thanks
So going by you reply, can i assume that if there is a configuration change to my schema I have to again index documents,
There is no short cut of updating the index.
Because we  cant afford to index 2 million documents again and again.
There should be some utility or command line which does this things in the background.

i hope i make sense.

darniz
Reply | Threaded
Open this post in threaded view
|

Re: initiate reindexing in solr for field type changes

Otis Gospodnetic-2
Darniz,

Yes, if there is an incompatible schema change, you need to reindex your documents.

Otis
P.S.
Please include the copy of the response when replying, so the context/background of your question is easy to figure out.
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR



----- Original Message ----

> From: darniz <[hidden email]>
> To: [hidden email]
> Sent: Wed, November 18, 2009 1:30:06 PM
> Subject: Re: initiate reindexing in solr for field type changes
>
>
> Thanks
> So going by you reply, can i assume that if there is a configuration change
> to my schema I have to again index documents,
> There is no short cut of updating the index.
> Because we  cant afford to index 2 million documents again and again.
> There should be some utility or command line which does this things in the
> background.
>
> i hope i make sense.
>
> darniz
> --
> View this message in context:
> http://old.nabble.com/initiate-reindexing-in-solr-for-field-type-changes-tp26397067p26413172.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: initiate reindexing in solr for field type changes

Darniz
Thanks
Could you elaborate what is compatible schema change.
Do you mean schema change which deals only with query time.

darniz

Otis Gospodnetic wrote
Darniz,

Yes, if there is an incompatible schema change, you need to reindex your documents.

Otis
P.S.
Please include the copy of the response when replying, so the context/background of your question is easy to figure out.
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR



----- Original Message ----
> From: darniz <rnizamuddin@edmunds.com>
> To: solr-user@lucene.apache.org
> Sent: Wed, November 18, 2009 1:30:06 PM
> Subject: Re: initiate reindexing in solr for field type changes
>
>
> Thanks
> So going by you reply, can i assume that if there is a configuration change
> to my schema I have to again index documents,
> There is no short cut of updating the index.
> Because we  cant afford to index 2 million documents again and again.
> There should be some utility or command line which does this things in the
> background.
>
> i hope i make sense.
>
> darniz
> --
> View this message in context:
> http://old.nabble.com/initiate-reindexing-in-solr-for-field-type-changes-tp26397067p26413172.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: initiate reindexing in solr for field type changes

Shalin Shekhar Mangar
On Thu, Nov 19, 2009 at 4:50 AM, darniz <[hidden email]> wrote:

>
> Thanks
> Could you elaborate what is compatible schema change.
> Do you mean schema change which deals only with query time.
>
>
A compatible schema change would be addition of new fields. Removal of
fields may also be called compatible as long as your application does not
try to index or query them.

Modifying the field type of an existing field or adding/removing/modifying
tokenizers or filters on a field type is usually an incompatible change and
needs re-indexing of affected documents.

--
Regards,
Shalin Shekhar Mangar.
Reply | Threaded
Open this post in threaded view
|

Re: initiate reindexing in solr for field type changes

Darniz
thanks
darniz

Shalin Shekhar Mangar wrote
On Thu, Nov 19, 2009 at 4:50 AM, darniz <rnizamuddin@edmunds.com> wrote:

>
> Thanks
> Could you elaborate what is compatible schema change.
> Do you mean schema change which deals only with query time.
>
>
A compatible schema change would be addition of new fields. Removal of
fields may also be called compatible as long as your application does not
try to index or query them.

Modifying the field type of an existing field or adding/removing/modifying
tokenizers or filters on a field type is usually an incompatible change and
needs re-indexing of affected documents.

--
Regards,
Shalin Shekhar Mangar.