SolrCore Initialization Failures

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

SolrCore Initialization Failures

shefalid
Hi,

I am using solr 7.2.1
Every now & then some of the cores fail to load due to "SolrCore
Initialization Failure".
From error message it looks like the reason for failure is the core missing
some of the files:

org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
Error opening new searcher
Caused by: java.nio.file.NoSuchFileException:
/opt/solr/volumes/data1/xxx-core-5/index/_xxx.cfe

But I could not find any reason for the loss of these files.
Also downloading the missing files from the backup we maintain resolves this
issue.
Any idea what might be causing this?

Thanks




--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: SolrCore Initialization Failures

Erick Erickson
This simply should _not_ be happening due to Solr/Lucene "doing
something". Some possibilities:
- some other process is deleting them
- somehow you have more than one Solr core pointing at the same data
directory. To do this you'd have had to do something like use
NoLockPolicy, so this is unlikely.
- you have errors you're not noticing like "disk full", which leads to
incomplete directories. This also sounds unlikely since you load from
backups so it sounds like an unchanging index. Since the segments are
read-only they shouldn't disappear due to disk full errors.

I've never heard of Solr/Lucene mysteriously deleting index files, so
I suspect something outside of Solr is doing this.

Best,
Erick

On Mon, Jun 11, 2018 at 11:25 AM, shefalid <[hidden email]> wrote:

> Hi,
>
> I am using solr 7.2.1
> Every now & then some of the cores fail to load due to "SolrCore
> Initialization Failure".
> From error message it looks like the reason for failure is the core missing
> some of the files:
>
> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> Error opening new searcher
> Caused by: java.nio.file.NoSuchFileException:
> /opt/solr/volumes/data1/xxx-core-5/index/_xxx.cfe
>
> But I could not find any reason for the loss of these files.
> Also downloading the missing files from the backup we maintain resolves this
> issue.
> Any idea what might be causing this?
>
> Thanks
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: SolrCore Initialization Failures

shefalid
Thanks for your response.

None of the processes are deleting any index files.
One data directory is only pointed by one core.

We are writing data at a high ingestion rate (100,000 records per second).
Commit happens once every 30 seconds.

Also a periodic service runs to backup the data to our backup data store.
This service starts by saving commit point & when its done, it released the
commit point.

Could this combination of these factors some how cause the deletion of
files?



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html