daily SolrCloud collection wipes

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

daily SolrCloud collection wipes

Werner Detter
Hi,

I've got a SolrCloud instance with two collections running (Solr 7.7.2) on
Debian Stretch VMs.

Every morning round about 03:30 the collection gets reset by $something and
I have no clue what causes this and how to prevent it as there are
even no log entries in SolrCloud (even with increased log level)

It seems like it's some internal trigger. For the sake of testings I've
completely recreated[1]the collections yesterday from scratch but they've
been reset again.

Maybe anybody else experienced something similar and can give a hint how to
track down the source of the collection resets

Thanks,
Werner


[1]
curl "http://localhost:8983/solr/admin/collections?action=DELETE&name=$SOLR_COLLECTION"
/bin/su - solr -c "/opt/solr/bin/solr create -c $SOLR_COLLECTION -s 1 -rf 2"
/bin/bash /opt/solr/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181 -cmd putfile /configs/$SOLR_COLLECTION/schema.xml /root/$SOLR_COLLECTION/conf/schema.xml
/bin/bash /opt/solr/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181 -cmd putfile /configs/$SOLR_COLLECTION/solrconfig.xml /root/$SOLR_COLLECTION/conf/solrconfig.xml
/usr/bin/curl "http://localhost:8983/solr/admin/collections?action=MODIFYCOLLECTION&collection=$SOLR_COLLECTION&maxShardsPerNode=1"

/usr/share/zookeeper/bin/zkCli.sh
rmr /configs/$SOLR_COLLECTION/managed-schema
Reply | Threaded
Open this post in threaded view
|

Re: daily SolrCloud collection wipes

Erick Erickson
There’s nothing in Solr that will wipe collections by itself outside of Time Routed Aliases, and there’s no indication that you’re using them. Even if you were, those should only delete collections past their expiration date, they’re pretty well tested to behave correctly.

This really looks like some external cron job, especially because there’s nothing in Solr that would be sending curl commands like this.

Best,
Erick

> On Nov 14, 2019, at 2:19 AM, Werner Detter <[hidden email]> wrote:
>
> Hi,
>
> I've got a SolrCloud instance with two collections running (Solr 7.7.2) on
> Debian Stretch VMs.
>
> Every morning round about 03:30 the collection gets reset by $something and
> I have no clue what causes this and how to prevent it as there are
> even no log entries in SolrCloud (even with increased log level)
>
> It seems like it's some internal trigger. For the sake of testings I've
> completely recreated[1]the collections yesterday from scratch but they've
> been reset again.
>
> Maybe anybody else experienced something similar and can give a hint how to
> track down the source of the collection resets
>
> Thanks,
> Werner
>
>
> [1]
> curl "http://localhost:8983/solr/admin/collections?action=DELETE&name=$SOLR_COLLECTION"
> /bin/su - solr -c "/opt/solr/bin/solr create -c $SOLR_COLLECTION -s 1 -rf 2"
> /bin/bash /opt/solr/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181 -cmd putfile /configs/$SOLR_COLLECTION/schema.xml /root/$SOLR_COLLECTION/conf/schema.xml
> /bin/bash /opt/solr/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181 -cmd putfile /configs/$SOLR_COLLECTION/solrconfig.xml /root/$SOLR_COLLECTION/conf/solrconfig.xml
> /usr/bin/curl "http://localhost:8983/solr/admin/collections?action=MODIFYCOLLECTION&collection=$SOLR_COLLECTION&maxShardsPerNode=1"
>
> /usr/share/zookeeper/bin/zkCli.sh
> rmr /configs/$SOLR_COLLECTION/managed-schema