Solr Cache clear

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Solr Cache clear

Lewin Joy (TMNA)
** PROTECTED 関係者外秘
How do I clear the solr caches without restarting Solr cluster?
Is there a way?
I tried reloading the collection. But, it did not help.

Thanks,
Lewin

Reply | Threaded
Open this post in threaded view
|

Re: Solr Cache clear

Shawn Heisey-2
On 4/8/2019 2:14 PM, Lewin Joy (TMNA) wrote:
> How do I clear the solr caches without restarting Solr cluster?
> Is there a way?
> I tried reloading the collection. But, it did not help.

When I reload a core on a test setup (solr 7.4.0), I see cache sizes reset.

What evidence are you seeing that reloading doesn't work?

Thanks,
Shawn
Reply | Threaded
Open this post in threaded view
|

Re: Solr Cache clear

Alexandre Rafalovitch
In reply to this post by Lewin Joy (TMNA)
You may have warming queries to prepopulate your cache. Check your
solrconfig.xml.

Regards,
    Alex

On Mon, Apr 8, 2019, 4:16 PM Lewin Joy (TMNA), <[hidden email]> wrote:

> ** PROTECTED 関係者外秘
> How do I clear the solr caches without restarting Solr cluster?
> Is there a way?
> I tried reloading the collection. But, it did not help.
>
> Thanks,
> Lewin
>
>
Reply | Threaded
Open this post in threaded view
|

RE: Solr Cache clear

Lewin Joy (TMNA)
In reply to this post by Shawn Heisey-2
Hmm. I only tried reloading the collection as a whole. Not the core reload.
Where do I see the cache sizes after reload?

-Lewin

-----Original Message-----
From: Shawn Heisey <[hidden email]>
Sent: Monday, April 8, 2019 5:10 PM
To: [hidden email]
Subject: Re: Solr Cache clear

On 4/8/2019 2:14 PM, Lewin Joy (TMNA) wrote:
> How do I clear the solr caches without restarting Solr cluster?
> Is there a way?
> I tried reloading the collection. But, it did not help.

When I reload a core on a test setup (solr 7.4.0), I see cache sizes reset.

What evidence are you seeing that reloading doesn't work?

Thanks,
Shawn
Reply | Threaded
Open this post in threaded view
|

RE: Solr Cache clear

Lewin Joy (TMNA)
In reply to this post by Alexandre Rafalovitch
Thank you for email, Alex.

I have the autowarmCount set as 0.
So, this shouldn't prepopulate with old cache data.

-Lewin

-----Original Message-----
From: Alexandre Rafalovitch <[hidden email]>
Sent: Monday, April 8, 2019 6:45 PM
To: solr-user <[hidden email]>
Subject: Re: Solr Cache clear

You may have warming queries to prepopulate your cache. Check your solrconfig.xml.

Regards,
    Alex

On Mon, Apr 8, 2019, 4:16 PM Lewin Joy (TMNA), <[hidden email]> wrote:

> ** PROTECTED 関係者外秘
> How do I clear the solr caches without restarting Solr cluster?
> Is there a way?
> I tried reloading the collection. But, it did not help.
>
> Thanks,
> Lewin
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Solr Cache clear

Walter Underwood
I’d like to know this, too. We run benchmarks with log replay, starting with warming queries, then a measurement run. It is a pain to to a rolling restart of the whole cluster before each benchmark run.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)

> On Apr 9, 2019, at 10:55 AM, Lewin Joy (TMNA) <[hidden email]> wrote:
>
> Thank you for email, Alex.
>
> I have the autowarmCount set as 0.
> So, this shouldn't prepopulate with old cache data.
>
> -Lewin
>
> -----Original Message-----
> From: Alexandre Rafalovitch <[hidden email]>
> Sent: Monday, April 8, 2019 6:45 PM
> To: solr-user <[hidden email]>
> Subject: Re: Solr Cache clear
>
> You may have warming queries to prepopulate your cache. Check your solrconfig.xml.
>
> Regards,
>    Alex
>
> On Mon, Apr 8, 2019, 4:16 PM Lewin Joy (TMNA), <[hidden email]> wrote:
>
>> ** PROTECTED 関係者外秘
>> How do I clear the solr caches without restarting Solr cluster?
>> Is there a way?
>> I tried reloading the collection. But, it did not help.
>>
>> Thanks,
>> Lewin
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Solr Cache clear

Shawn Heisey-2
In reply to this post by Lewin Joy (TMNA)
On 4/9/2019 11:51 AM, Lewin Joy (TMNA) wrote:
> Hmm. I only tried reloading the collection as a whole. Not the core reload.
> Where do I see the cache sizes after reload?

If you do not know how to see the cache sizes, then what information are
you looking at which has led you to the conclusion that the caches have
not been cleared?

To get to cache stats:  In the admin UI, choose a core from the
dropdown.  Then click on Plugins/Stats, then CACHE, and choose the cache
you want to look at.

Thanks,
Shawn
Reply | Threaded
Open this post in threaded view
|

RE: Solr Cache clear

Lewin Joy (TMNA)
Hi Shawn,

We are facing an issue where the caches got corrupted.
We are doing a json.facet and pivoting through 3 levels. We are taking allBuckets from the different levels.

In json.facet query, while doing the inner facets, we are keeping a limit. We notice that as we change the limit, we are getting a different value for allBuckets.
And this got corrected after I explicitly applied one facet value as a filter one time. I am assuming, it cleared the cache for that filter.

Now, I have few other facet values having the similar issue. Assuming, that this issue would get resolved if I clear the cache, I am checking these values once I reload the collection.

Anyway, If I am able to look at the cache sizes after reload, this gives me more information.

I just tried to go to the location you have specified. I could not see a "CACHE" . I can see the "Statistics" section.
I am using Solr 7.2 on solrcloud mode.

thanks
-Lewin

-----Original Message-----
From: Shawn Heisey <[hidden email]>
Sent: Tuesday, April 9, 2019 1:01 PM
To: [hidden email]
Subject: Re: Solr Cache clear

On 4/9/2019 11:51 AM, Lewin Joy (TMNA) wrote:
> Hmm. I only tried reloading the collection as a whole. Not the core reload.
> Where do I see the cache sizes after reload?

If you do not know how to see the cache sizes, then what information are you looking at which has led you to the conclusion that the caches have not been cleared?

To get to cache stats:  In the admin UI, choose a core from the dropdown.  Then click on Plugins/Stats, then CACHE, and choose the cache you want to look at.

Thanks,
Shawn
Reply | Threaded
Open this post in threaded view
|

Re: Solr Cache clear

Shawn Heisey-2
On 4/9/2019 12:38 PM, Lewin Joy (TMNA) wrote:
> I just tried to go to the location you have specified. I could not see a "CACHE" . I can see the "Statistics" section.
> I am using Solr 7.2 on solrcloud mode.

If you are trying to select a *collection* from a dropdown, you will not
see this.  It will only show up when you select a *core* from the other
dropdown.

In SolrCloud, collections are made up of one or more shards.  Shards are
made up of one or more replicas.  Every shard replica is a core.

Here are some partial screenshots showing what I clicked on to get to
the cache stats:

https://www.dropbox.com/s/ked4xe4w45e9qr2/cache-1.png?dl=0
https://www.dropbox.com/s/7v630mr2ii0rtee/cache-2.png?dl=0
https://www.dropbox.com/s/6czqtxq5qf8vzwz/cache-3.png?dl=0

The system where I captured these screenshots was not running in
SolrCloud mode, so it did not have the "collections" dropdown that your
admin UI will have.

When you reload a collection, Solr uses the information in ZooKeeper to
locate all of the shard replicas that make up the collection, and
reloads all those cores.  So a collection reload is basically equivalent
to multiple core reloads.

Thanks,
Shawn
Reply | Threaded
Open this post in threaded view
|

RE: Solr Cache clear

Lewin Joy (TMNA)
Hmm. I am doing the same thing. But, somehow in my browser, after I select the core, it does not stay selected to view the stats/cache.
Attaching the gif for when I try it.

Anyway, that is a different issue from my side. Thanks for your input.

-Lewin

-----Original Message-----
From: Shawn Heisey <[hidden email]>
Sent: Tuesday, April 9, 2019 1:52 PM
To: [hidden email]
Subject: Re: Solr Cache clear

On 4/9/2019 12:38 PM, Lewin Joy (TMNA) wrote:
> I just tried to go to the location you have specified. I could not see a "CACHE" . I can see the "Statistics" section.
> I am using Solr 7.2 on solrcloud mode.

If you are trying to select a *collection* from a dropdown, you will not see this.  It will only show up when you select a *core* from the other dropdown.

In SolrCloud, collections are made up of one or more shards.  Shards are made up of one or more replicas.  Every shard replica is a core.

Here are some partial screenshots showing what I clicked on to get to the cache stats:

https://urldefense.proofpoint.com/v2/url?u=https-3A__www.dropbox.com_s_ked4xe4w45e9qr2_cache-2D1.png-3Fdl-3D0&d=DwICaQ&c=DDPRwrN9uYSNUDpKqPeD1g&r=WMeiuwk_Qf7aOundlWmtZMlairjO8ZQxQpAndx7JD6A&m=9o8kD9YOypvijGkYVvikg-74p2wDw2kblxyrVUijnaI&s=Zr1cZF633BlxWx0semmI-DCywSyOteEEM9_eABk-OrM&e=
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.dropbox.com_s_7v630mr2ii0rtee_cache-2D2.png-3Fdl-3D0&d=DwICaQ&c=DDPRwrN9uYSNUDpKqPeD1g&r=WMeiuwk_Qf7aOundlWmtZMlairjO8ZQxQpAndx7JD6A&m=9o8kD9YOypvijGkYVvikg-74p2wDw2kblxyrVUijnaI&s=rCCFdR6yk2ElhXMyxX0LJJoPkymG8Y3rrxCQz3gK8fI&e=
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.dropbox.com_s_6czqtxq5qf8vzwz_cache-2D3.png-3Fdl-3D0&d=DwICaQ&c=DDPRwrN9uYSNUDpKqPeD1g&r=WMeiuwk_Qf7aOundlWmtZMlairjO8ZQxQpAndx7JD6A&m=9o8kD9YOypvijGkYVvikg-74p2wDw2kblxyrVUijnaI&s=sA5FxVsaP1M8bGkxUnGdVoW7nLjF7rFthVCG6sKY_po&e=

The system where I captured these screenshots was not running in SolrCloud mode, so it did not have the "collections" dropdown that your admin UI will have.

When you reload a collection, Solr uses the information in ZooKeeper to locate all of the shard replicas that make up the collection, and reloads all those cores.  So a collection reload is basically equivalent to multiple core reloads.

Thanks,
Shawn