Questions about corrupted Segments files.

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Questions about corrupted Segments files.

Kayak28
Hello, Community members:

I am using Solr 7.7.2.
On the other day, while indexing to the Solr, my computer powered off.
As a result, there are corrupted segment files.

Is there any way to fix the corrupted segment files without re-indexing?

I have read a blog post (in Japanese) writing about checkIndex method which
can be used to determine/fix corrupted segment files, but when I tried to
run the following command, I got the error message.
So, I am not sure if checkIndex can actually fix the index files.


java -cp lucene-core-7.7.2.jar -ea:org.apache.lucene...
org.apache.lucene.index.CheckIndex solr/server/solr/basic_copy/data/index
-fix


ERROR: unexpected extra argument '-fix'



If anybody knows about either a way to fix corrupted segment files or a way
to use checkIndex '-fix' option correctly, could you please let me know?

Any clue will be very appreciated.

Sincerely,
Kaya Ota
Reply | Threaded
Open this post in threaded view
|

Re: Questions about corrupted Segments files.

Erick Erickson
Well, the fix option “fixes” the index by deleting any segments it can’t read, thus also deleting the documents in that segment. So if you can get it to run, it’ll allow you to continue using the index, but you’ll have to re-index all your data anyway to get the documents in the deleted segments back. It’s intended as a stopgap, not a true repair.

-fix has been renamed -exorcise to emphasize that it has serious consequences, see: https://issues.apache.org/jira/browse/LUCENE-5924

Best,
Erick

> On Nov 5, 2019, at 8:24 PM, Kayak28 <[hidden email]> wrote:
>
> Hello, Community members:
>
> I am using Solr 7.7.2.
> On the other day, while indexing to the Solr, my computer powered off.
> As a result, there are corrupted segment files.
>
> Is there any way to fix the corrupted segment files without re-indexing?
>
> I have read a blog post (in Japanese) writing about checkIndex method which
> can be used to determine/fix corrupted segment files, but when I tried to
> run the following command, I got the error message.
> So, I am not sure if checkIndex can actually fix the index files.
>
>
> java -cp lucene-core-7.7.2.jar -ea:org.apache.lucene...
> org.apache.lucene.index.CheckIndex solr/server/solr/basic_copy/data/index
> -fix
>
>
> ERROR: unexpected extra argument '-fix'
>
>
>
> If anybody knows about either a way to fix corrupted segment files or a way
> to use checkIndex '-fix' option correctly, could you please let me know?
>
> Any clue will be very appreciated.
>
> Sincerely,
> Kaya Ota

Reply | Threaded
Open this post in threaded view
|

Re: Questions about corrupted Segments files.

Kayak28
Hello, Mr. Erick Erickson and Community Members:

Thank you for your quick response.

I understand that "checkIndex -exorcise" does not actually fix the indices,
more like "first aid" kinda action.
And soon or later I need to re-index from my source.

What is the rule of thumb to recover from the corrupted segment files?

Sincerely,
Kaya Ota






2019年11月6日(水) 10:56 Erick Erickson <[hidden email]>:

> Well, the fix option “fixes” the index by deleting any segments it can’t
> read, thus also deleting the documents in that segment. So if you can get
> it to run, it’ll allow you to continue using the index, but you’ll have to
> re-index all your data anyway to get the documents in the deleted segments
> back. It’s intended as a stopgap, not a true repair.
>
> -fix has been renamed -exorcise to emphasize that it has serious
> consequences, see: https://issues.apache.org/jira/browse/LUCENE-5924
>
> Best,
> Erick
>
> > On Nov 5, 2019, at 8:24 PM, Kayak28 <[hidden email]> wrote:
> >
> > Hello, Community members:
> >
> > I am using Solr 7.7.2.
> > On the other day, while indexing to the Solr, my computer powered off.
> > As a result, there are corrupted segment files.
> >
> > Is there any way to fix the corrupted segment files without re-indexing?
> >
> > I have read a blog post (in Japanese) writing about checkIndex method
> which
> > can be used to determine/fix corrupted segment files, but when I tried to
> > run the following command, I got the error message.
> > So, I am not sure if checkIndex can actually fix the index files.
> >
> >
> > java -cp lucene-core-7.7.2.jar -ea:org.apache.lucene...
> > org.apache.lucene.index.CheckIndex solr/server/solr/basic_copy/data/index
> > -fix
> >
> >
> > ERROR: unexpected extra argument '-fix'
> >
> >
> >
> > If anybody knows about either a way to fix corrupted segment files or a
> way
> > to use checkIndex '-fix' option correctly, could you please let me know?
> >
> > Any clue will be very appreciated.
> >
> > Sincerely,
> > Kaya Ota
>
>