Memory line in status output

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Memory line in status output

Ryan W
Hi all,

What is the meaning of the "memory" line in the output when I run the solr
status command?  What controls whether that memory gets exhausted?  At
times if I run "solr status" over and over, that memory number creeps up
and up and up.  Presumably it is not a good thing if it moves all the way
up to my 31GB capacity.  What controls whether that happens?  How do I
prevent that?  Or does Solr manage this automatically?


$ /opt/solr/bin/solr status

Found 1 Solr nodes:

Solr process 101530 running on port 8983
{
  "solr_home":"/opt/solr/server/solr",
  "version":"7.7.2 d4c30fc2856154f2c1fefc589eb7cd070a415b94 - janhoy -
2019-05-28 23:37:48",
  "startTime":"2020-10-12T12:04:57.379Z",
  "uptime":"0 days, 1 hours, 46 minutes, 41 seconds",
  "memory":"3.3 GB (%10.7) of 31 GB"}
Reply | Threaded
Open this post in threaded view
|

Re: Memory line in status output

Erick Erickson
Solr doesn’t manage this at all, it’s the JVM’s garbage collection
that occasionally kicks in. In general, memory creeps up until
the GC threshold is set (which there are about a zillion
parameters that you can set) and then GC kicks in.

Generally, the recommendation is to use the G1GC collector
and just leave the default settings as they are.

It’s usually a mistake, BTW, to over-allocate memory. You should shrink the
heap as far as you can and still maintain a reasonable safety margin. See:

https://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html

What’s a “reasonable safety margin”? Unfortunately you have to experiment.

Best,
Erick

> On Oct 12, 2020, at 10:33 AM, Ryan W <[hidden email]> wrote:
>
> Hi all,
>
> What is the meaning of the "memory" line in the output when I run the solr
> status command?  What controls whether that memory gets exhausted?  At
> times if I run "solr status" over and over, that memory number creeps up
> and up and up.  Presumably it is not a good thing if it moves all the way
> up to my 31GB capacity.  What controls whether that happens?  How do I
> prevent that?  Or does Solr manage this automatically?
>
>
> $ /opt/solr/bin/solr status
>
> Found 1 Solr nodes:
>
> Solr process 101530 running on port 8983
> {
>  "solr_home":"/opt/solr/server/solr",
>  "version":"7.7.2 d4c30fc2856154f2c1fefc589eb7cd070a415b94 - janhoy -
> 2019-05-28 23:37:48",
>  "startTime":"2020-10-12T12:04:57.379Z",
>  "uptime":"0 days, 1 hours, 46 minutes, 41 seconds",
>  "memory":"3.3 GB (%10.7) of 31 GB"}

Reply | Threaded
Open this post in threaded view
|

Re: Memory line in status output

Ryan W
Thanks.  How do I activate the G1GC collector?  Do I do this by editing a
config file, or by adding a parameter when I start solr?

Oracle's docs are pointing me to a file that supposedly is at
instance-dir/OUD/config/java.properties, but I don't have that path.  I am
not sure what is meant by instance-dir here, but perhaps it means my JRE
install, which is at
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.262.b10-0.el7_8.x86_64/jre -- but
there is no "OUD" directory in this location.



On Mon, Oct 12, 2020 at 11:15 AM Erick Erickson <[hidden email]>
wrote:

> Solr doesn’t manage this at all, it’s the JVM’s garbage collection
> that occasionally kicks in. In general, memory creeps up until
> the GC threshold is set (which there are about a zillion
> parameters that you can set) and then GC kicks in.
>
> Generally, the recommendation is to use the G1GC collector
> and just leave the default settings as they are.
>
> It’s usually a mistake, BTW, to over-allocate memory. You should shrink the
> heap as far as you can and still maintain a reasonable safety margin. See:
>
> https://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
>
> What’s a “reasonable safety margin”? Unfortunately you have to experiment.
>
> Best,
> Erick
>
> > On Oct 12, 2020, at 10:33 AM, Ryan W <[hidden email]> wrote:
> >
> > Hi all,
> >
> > What is the meaning of the "memory" line in the output when I run the
> solr
> > status command?  What controls whether that memory gets exhausted?  At
> > times if I run "solr status" over and over, that memory number creeps up
> > and up and up.  Presumably it is not a good thing if it moves all the way
> > up to my 31GB capacity.  What controls whether that happens?  How do I
> > prevent that?  Or does Solr manage this automatically?
> >
> >
> > $ /opt/solr/bin/solr status
> >
> > Found 1 Solr nodes:
> >
> > Solr process 101530 running on port 8983
> > {
> >  "solr_home":"/opt/solr/server/solr",
> >  "version":"7.7.2 d4c30fc2856154f2c1fefc589eb7cd070a415b94 - janhoy -
> > 2019-05-28 23:37:48",
> >  "startTime":"2020-10-12T12:04:57.379Z",
> >  "uptime":"0 days, 1 hours, 46 minutes, 41 seconds",
> >  "memory":"3.3 GB (%10.7) of 31 GB"}
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Memory line in status output

Shawn Heisey
On 10/12/2020 5:11 PM, Ryan W wrote:
> Thanks.  How do I activate the G1GC collector?  Do I do this by editing a
> config file, or by adding a parameter when I start solr?
>
> Oracle's docs are pointing me to a file that supposedly is at
> instance-dir/OUD/config/java.properties, but I don't have that path.  I am
> not sure what is meant by instance-dir here, but perhaps it means my JRE
> install, which is at
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.262.b10-0.el7_8.x86_64/jre -- but
> there is no "OUD" directory in this location.

The collector is chosen by the startup options given to Java, in this
case by the start script for Solr.  I've never heard of it being set by
a config in the JRE.

In Solr 7, the start script defaults to the CMS collector.  We have
updated that to G1 in the latest Solr 8.x versions, because CMS has been
deprecated by Oracle.

Adding the following lines to the correct solr.in.sh would change the
garbage collector to G1.  I got this from the "bin/solr" script in Solr
8.5.1:

       GC_TUNE=('-XX:+UseG1GC' \
         '-XX:+PerfDisableSharedMem' \
         '-XX:+ParallelRefProcEnabled' \
         '-XX:MaxGCPauseMillis=250' \
         '-XX:+UseLargePages' \
         '-XX:+AlwaysPreTouch')

If you used the service installer script to install Solr, then the
correct file to add this to is usually /etc/default/solr.in.sh ... but
if you did the install manually, it may be in the same bin directory
that contains the solr script itself.  Your initial message says the
solr home is /opt/solr/server/solr so I am assuming it's not running on
Windows.

Thanks,
Shawn
Reply | Threaded
Open this post in threaded view
|

Re: Memory line in status output

Ryan W
This is how I start solr:

/opt/solr/bin/solr start

In my /etc/default/solr.in.sh, I have this...

GC_TUNE=" \
-XX:+UseG1GC \
-XX:+ParallelRefProcEnabled \
-XX:G1HeapRegionSize=8m \
-XX:MaxGCPauseMillis=200 \
-XX:+UseLargePages \
-XX:+AggressiveOpts \
"

But I don't know how to tell if Solr is using that file.

In my /opt/solr/bin there is no solr.in.sh, but there is a solr.in.sh.orig
-- perhaps I should copy my /etc/default/solr.in.sh to /opt/solr/bin ?

I am running Linux (RHEL).  The Solr version is 7.7.2.  Solr 8.x is not
compatible with my application.

Thank you.


On Mon, Oct 12, 2020 at 9:46 PM Shawn Heisey <[hidden email]> wrote:

> On 10/12/2020 5:11 PM, Ryan W wrote:
> > Thanks.  How do I activate the G1GC collector?  Do I do this by editing a
> > config file, or by adding a parameter when I start solr?
> >
> > Oracle's docs are pointing me to a file that supposedly is at
> > instance-dir/OUD/config/java.properties, but I don't have that path.  I
> am
> > not sure what is meant by instance-dir here, but perhaps it means my JRE
> > install, which is at
> > /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.262.b10-0.el7_8.x86_64/jre -- but
> > there is no "OUD" directory in this location.
>
> The collector is chosen by the startup options given to Java, in this
> case by the start script for Solr.  I've never heard of it being set by
> a config in the JRE.
>
> In Solr 7, the start script defaults to the CMS collector.  We have
> updated that to G1 in the latest Solr 8.x versions, because CMS has been
> deprecated by Oracle.
>
> Adding the following lines to the correct solr.in.sh would change the
> garbage collector to G1.  I got this from the "bin/solr" script in Solr
> 8.5.1:
>
>        GC_TUNE=('-XX:+UseG1GC' \
>          '-XX:+PerfDisableSharedMem' \
>          '-XX:+ParallelRefProcEnabled' \
>          '-XX:MaxGCPauseMillis=250' \
>          '-XX:+UseLargePages' \
>          '-XX:+AlwaysPreTouch')
>
> If you used the service installer script to install Solr, then the
> correct file to add this to is usually /etc/default/solr.in.sh ... but
> if you did the install manually, it may be in the same bin directory
> that contains the solr script itself.  Your initial message says the
> solr home is /opt/solr/server/solr so I am assuming it's not running on
> Windows.
>
> Thanks,
> Shawn
>
Reply | Threaded
Open this post in threaded view
|

Re: Memory line in status output

Ryan W
Or, perhaps if I start solr like so....

service solr start

...it will use the solr.in.sh at /etc/default/solr.in.sh ?



On Tue, Oct 13, 2020 at 11:19 AM Ryan W <[hidden email]> wrote:

> This is how I start solr:
>
> /opt/solr/bin/solr start
>
> In my /etc/default/solr.in.sh, I have this...
>
> GC_TUNE=" \
> -XX:+UseG1GC \
> -XX:+ParallelRefProcEnabled \
> -XX:G1HeapRegionSize=8m \
> -XX:MaxGCPauseMillis=200 \
> -XX:+UseLargePages \
> -XX:+AggressiveOpts \
> "
>
> But I don't know how to tell if Solr is using that file.
>
> In my /opt/solr/bin there is no solr.in.sh, but there is a
> solr.in.sh.orig -- perhaps I should copy my /etc/default/solr.in.sh to
> /opt/solr/bin ?
>
> I am running Linux (RHEL).  The Solr version is 7.7.2.  Solr 8.x is not
> compatible with my application.
>
> Thank you.
>
>
> On Mon, Oct 12, 2020 at 9:46 PM Shawn Heisey <[hidden email]>
> wrote:
>
>> On 10/12/2020 5:11 PM, Ryan W wrote:
>> > Thanks.  How do I activate the G1GC collector?  Do I do this by editing
>> a
>> > config file, or by adding a parameter when I start solr?
>> >
>> > Oracle's docs are pointing me to a file that supposedly is at
>> > instance-dir/OUD/config/java.properties, but I don't have that path.  I
>> am
>> > not sure what is meant by instance-dir here, but perhaps it means my JRE
>> > install, which is at
>> > /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.262.b10-0.el7_8.x86_64/jre -- but
>> > there is no "OUD" directory in this location.
>>
>> The collector is chosen by the startup options given to Java, in this
>> case by the start script for Solr.  I've never heard of it being set by
>> a config in the JRE.
>>
>> In Solr 7, the start script defaults to the CMS collector.  We have
>> updated that to G1 in the latest Solr 8.x versions, because CMS has been
>> deprecated by Oracle.
>>
>> Adding the following lines to the correct solr.in.sh would change the
>> garbage collector to G1.  I got this from the "bin/solr" script in Solr
>> 8.5.1:
>>
>>        GC_TUNE=('-XX:+UseG1GC' \
>>          '-XX:+PerfDisableSharedMem' \
>>          '-XX:+ParallelRefProcEnabled' \
>>          '-XX:MaxGCPauseMillis=250' \
>>          '-XX:+UseLargePages' \
>>          '-XX:+AlwaysPreTouch')
>>
>> If you used the service installer script to install Solr, then the
>> correct file to add this to is usually /etc/default/solr.in.sh ... but
>> if you did the install manually, it may be in the same bin directory
>> that contains the solr script itself.  Your initial message says the
>> solr home is /opt/solr/server/solr so I am assuming it's not running on
>> Windows.
>>
>> Thanks,
>> Shawn
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Memory line in status output

Ryan W
I think I have it sorted. At this point I'm using GCG1, I take it, because
most recently I started Solr as a service...

service solr start

And that is running solr by way of /etc/init.d/solr because I don't have
any systemd unit for solr, as explained here...
https://askubuntu.com/questions/903354/difference-between-systemctl-and-service-commands

And I can see in the System V script for solr that /etc/default/solr.in.sh
is the relevant config file.


On Tue, Oct 13, 2020 at 11:23 AM Ryan W <[hidden email]> wrote:

> Or, perhaps if I start solr like so....
>
> service solr start
>
> ...it will use the solr.in.sh at /etc/default/solr.in.sh ?
>
>
>
> On Tue, Oct 13, 2020 at 11:19 AM Ryan W <[hidden email]> wrote:
>
>> This is how I start solr:
>>
>> /opt/solr/bin/solr start
>>
>> In my /etc/default/solr.in.sh, I have this...
>>
>> GC_TUNE=" \
>> -XX:+UseG1GC \
>> -XX:+ParallelRefProcEnabled \
>> -XX:G1HeapRegionSize=8m \
>> -XX:MaxGCPauseMillis=200 \
>> -XX:+UseLargePages \
>> -XX:+AggressiveOpts \
>> "
>>
>> But I don't know how to tell if Solr is using that file.
>>
>> In my /opt/solr/bin there is no solr.in.sh, but there is a
>> solr.in.sh.orig -- perhaps I should copy my /etc/default/solr.in.sh to
>> /opt/solr/bin ?
>>
>> I am running Linux (RHEL).  The Solr version is 7.7.2.  Solr 8.x is not
>> compatible with my application.
>>
>> Thank you.
>>
>>
>> On Mon, Oct 12, 2020 at 9:46 PM Shawn Heisey <[hidden email]>
>> wrote:
>>
>>> On 10/12/2020 5:11 PM, Ryan W wrote:
>>> > Thanks.  How do I activate the G1GC collector?  Do I do this by
>>> editing a
>>> > config file, or by adding a parameter when I start solr?
>>> >
>>> > Oracle's docs are pointing me to a file that supposedly is at
>>> > instance-dir/OUD/config/java.properties, but I don't have that path.
>>> I am
>>> > not sure what is meant by instance-dir here, but perhaps it means my
>>> JRE
>>> > install, which is at
>>> > /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.262.b10-0.el7_8.x86_64/jre -- but
>>> > there is no "OUD" directory in this location.
>>>
>>> The collector is chosen by the startup options given to Java, in this
>>> case by the start script for Solr.  I've never heard of it being set by
>>> a config in the JRE.
>>>
>>> In Solr 7, the start script defaults to the CMS collector.  We have
>>> updated that to G1 in the latest Solr 8.x versions, because CMS has been
>>> deprecated by Oracle.
>>>
>>> Adding the following lines to the correct solr.in.sh would change the
>>> garbage collector to G1.  I got this from the "bin/solr" script in Solr
>>> 8.5.1:
>>>
>>>        GC_TUNE=('-XX:+UseG1GC' \
>>>          '-XX:+PerfDisableSharedMem' \
>>>          '-XX:+ParallelRefProcEnabled' \
>>>          '-XX:MaxGCPauseMillis=250' \
>>>          '-XX:+UseLargePages' \
>>>          '-XX:+AlwaysPreTouch')
>>>
>>> If you used the service installer script to install Solr, then the
>>> correct file to add this to is usually /etc/default/solr.in.sh ... but
>>> if you did the install manually, it may be in the same bin directory
>>> that contains the solr script itself.  Your initial message says the
>>> solr home is /opt/solr/server/solr so I am assuming it's not running on
>>> Windows.
>>>
>>> Thanks,
>>> Shawn
>>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: Memory line in status output

Walter Underwood
The home page of the Solr admin UI shows all of the options to the JVM.
That will include the choice of garbage collector.

You can also see the options with “ps -ef | grep solr”.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)

> On Oct 13, 2020, at 1:19 PM, Ryan W <[hidden email]> wrote:
>
> I think I have it sorted. At this point I'm using GCG1, I take it, because
> most recently I started Solr as a service...
>
> service solr start
>
> And that is running solr by way of /etc/init.d/solr because I don't have
> any systemd unit for solr, as explained here...
> https://askubuntu.com/questions/903354/difference-between-systemctl-and-service-commands
>
> And I can see in the System V script for solr that /etc/default/solr.in.sh
> is the relevant config file.
>
>
> On Tue, Oct 13, 2020 at 11:23 AM Ryan W <[hidden email]> wrote:
>
>> Or, perhaps if I start solr like so....
>>
>> service solr start
>>
>> ...it will use the solr.in.sh at /etc/default/solr.in.sh ?
>>
>>
>>
>> On Tue, Oct 13, 2020 at 11:19 AM Ryan W <[hidden email]> wrote:
>>
>>> This is how I start solr:
>>>
>>> /opt/solr/bin/solr start
>>>
>>> In my /etc/default/solr.in.sh, I have this...
>>>
>>> GC_TUNE=" \
>>> -XX:+UseG1GC \
>>> -XX:+ParallelRefProcEnabled \
>>> -XX:G1HeapRegionSize=8m \
>>> -XX:MaxGCPauseMillis=200 \
>>> -XX:+UseLargePages \
>>> -XX:+AggressiveOpts \
>>> "
>>>
>>> But I don't know how to tell if Solr is using that file.
>>>
>>> In my /opt/solr/bin there is no solr.in.sh, but there is a
>>> solr.in.sh.orig -- perhaps I should copy my /etc/default/solr.in.sh to
>>> /opt/solr/bin ?
>>>
>>> I am running Linux (RHEL).  The Solr version is 7.7.2.  Solr 8.x is not
>>> compatible with my application.
>>>
>>> Thank you.
>>>
>>>
>>> On Mon, Oct 12, 2020 at 9:46 PM Shawn Heisey <[hidden email]>
>>> wrote:
>>>
>>>> On 10/12/2020 5:11 PM, Ryan W wrote:
>>>>> Thanks.  How do I activate the G1GC collector?  Do I do this by
>>>> editing a
>>>>> config file, or by adding a parameter when I start solr?
>>>>>
>>>>> Oracle's docs are pointing me to a file that supposedly is at
>>>>> instance-dir/OUD/config/java.properties, but I don't have that path.
>>>> I am
>>>>> not sure what is meant by instance-dir here, but perhaps it means my
>>>> JRE
>>>>> install, which is at
>>>>> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.262.b10-0.el7_8.x86_64/jre -- but
>>>>> there is no "OUD" directory in this location.
>>>>
>>>> The collector is chosen by the startup options given to Java, in this
>>>> case by the start script for Solr.  I've never heard of it being set by
>>>> a config in the JRE.
>>>>
>>>> In Solr 7, the start script defaults to the CMS collector.  We have
>>>> updated that to G1 in the latest Solr 8.x versions, because CMS has been
>>>> deprecated by Oracle.
>>>>
>>>> Adding the following lines to the correct solr.in.sh would change the
>>>> garbage collector to G1.  I got this from the "bin/solr" script in Solr
>>>> 8.5.1:
>>>>
>>>>       GC_TUNE=('-XX:+UseG1GC' \
>>>>         '-XX:+PerfDisableSharedMem' \
>>>>         '-XX:+ParallelRefProcEnabled' \
>>>>         '-XX:MaxGCPauseMillis=250' \
>>>>         '-XX:+UseLargePages' \
>>>>         '-XX:+AlwaysPreTouch')
>>>>
>>>> If you used the service installer script to install Solr, then the
>>>> correct file to add this to is usually /etc/default/solr.in.sh ... but
>>>> if you did the install manually, it may be in the same bin directory
>>>> that contains the solr script itself.  Your initial message says the
>>>> solr home is /opt/solr/server/solr so I am assuming it's not running on
>>>> Windows.
>>>>
>>>> Thanks,
>>>> Shawn
>>>>
>>>

Reply | Threaded
Open this post in threaded view
|

Re: Memory line in status output

Ryan W
Thanks.  The G1 docs say "G1 is designed to provide good overall
performance without the need to specify additional options."

Would that look like this...

GC_TUNE=" \
-XX:+UseG1GC \
"

Is that the most minimal config? Is it typical to use it without options?

On Tue, Oct 13, 2020 at 4:22 PM Walter Underwood <[hidden email]>
wrote:

> The home page of the Solr admin UI shows all of the options to the JVM.
> That will include the choice of garbage collector.
>
> You can also see the options with “ps -ef | grep solr”.
>
> wunder
> Walter Underwood
> [hidden email]
> http://observer.wunderwood.org/  (my blog)
>
> > On Oct 13, 2020, at 1:19 PM, Ryan W <[hidden email]> wrote:
> >
> > I think I have it sorted. At this point I'm using GCG1, I take it,
> because
> > most recently I started Solr as a service...
> >
> > service solr start
> >
> > And that is running solr by way of /etc/init.d/solr because I don't have
> > any systemd unit for solr, as explained here...
> >
> https://askubuntu.com/questions/903354/difference-between-systemctl-and-service-commands
> >
> > And I can see in the System V script for solr that /etc/default/
> solr.in.sh
> > is the relevant config file.
> >
> >
> > On Tue, Oct 13, 2020 at 11:23 AM Ryan W <[hidden email]> wrote:
> >
> >> Or, perhaps if I start solr like so....
> >>
> >> service solr start
> >>
> >> ...it will use the solr.in.sh at /etc/default/solr.in.sh ?
> >>
> >>
> >>
> >> On Tue, Oct 13, 2020 at 11:19 AM Ryan W <[hidden email]> wrote:
> >>
> >>> This is how I start solr:
> >>>
> >>> /opt/solr/bin/solr start
> >>>
> >>> In my /etc/default/solr.in.sh, I have this...
> >>>
> >>> GC_TUNE=" \
> >>> -XX:+UseG1GC \
> >>> -XX:+ParallelRefProcEnabled \
> >>> -XX:G1HeapRegionSize=8m \
> >>> -XX:MaxGCPauseMillis=200 \
> >>> -XX:+UseLargePages \
> >>> -XX:+AggressiveOpts \
> >>> "
> >>>
> >>> But I don't know how to tell if Solr is using that file.
> >>>
> >>> In my /opt/solr/bin there is no solr.in.sh, but there is a
> >>> solr.in.sh.orig -- perhaps I should copy my /etc/default/solr.in.sh to
> >>> /opt/solr/bin ?
> >>>
> >>> I am running Linux (RHEL).  The Solr version is 7.7.2.  Solr 8.x is not
> >>> compatible with my application.
> >>>
> >>> Thank you.
> >>>
> >>>
> >>> On Mon, Oct 12, 2020 at 9:46 PM Shawn Heisey <[hidden email]>
> >>> wrote:
> >>>
> >>>> On 10/12/2020 5:11 PM, Ryan W wrote:
> >>>>> Thanks.  How do I activate the G1GC collector?  Do I do this by
> >>>> editing a
> >>>>> config file, or by adding a parameter when I start solr?
> >>>>>
> >>>>> Oracle's docs are pointing me to a file that supposedly is at
> >>>>> instance-dir/OUD/config/java.properties, but I don't have that path.
> >>>> I am
> >>>>> not sure what is meant by instance-dir here, but perhaps it means my
> >>>> JRE
> >>>>> install, which is at
> >>>>> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.262.b10-0.el7_8.x86_64/jre --
> but
> >>>>> there is no "OUD" directory in this location.
> >>>>
> >>>> The collector is chosen by the startup options given to Java, in this
> >>>> case by the start script for Solr.  I've never heard of it being set
> by
> >>>> a config in the JRE.
> >>>>
> >>>> In Solr 7, the start script defaults to the CMS collector.  We have
> >>>> updated that to G1 in the latest Solr 8.x versions, because CMS has
> been
> >>>> deprecated by Oracle.
> >>>>
> >>>> Adding the following lines to the correct solr.in.sh would change the
> >>>> garbage collector to G1.  I got this from the "bin/solr" script in
> Solr
> >>>> 8.5.1:
> >>>>
> >>>>       GC_TUNE=('-XX:+UseG1GC' \
> >>>>         '-XX:+PerfDisableSharedMem' \
> >>>>         '-XX:+ParallelRefProcEnabled' \
> >>>>         '-XX:MaxGCPauseMillis=250' \
> >>>>         '-XX:+UseLargePages' \
> >>>>         '-XX:+AlwaysPreTouch')
> >>>>
> >>>> If you used the service installer script to install Solr, then the
> >>>> correct file to add this to is usually /etc/default/solr.in.sh ...
> but
> >>>> if you did the install manually, it may be in the same bin directory
> >>>> that contains the solr script itself.  Your initial message says the
> >>>> solr home is /opt/solr/server/solr so I am assuming it's not running
> on
> >>>> Windows.
> >>>>
> >>>> Thanks,
> >>>> Shawn
> >>>>
> >>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Memory line in status output

Walter Underwood
I recommend using the options mentioned in recent messages on this list.

Solr has pretty specific memory demands, with lots of allocations with a
lifetime of a single request, plus very long-lived allocations that aren’t freed
until they are evicted from a cache.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)

> On Oct 13, 2020, at 1:35 PM, Ryan W <[hidden email]> wrote:
>
> Thanks.  The G1 docs say "G1 is designed to provide good overall
> performance without the need to specify additional options."
>
> Would that look like this...
>
> GC_TUNE=" \
> -XX:+UseG1GC \
> "
>
> Is that the most minimal config? Is it typical to use it without options?
>
> On Tue, Oct 13, 2020 at 4:22 PM Walter Underwood <[hidden email]>
> wrote:
>
>> The home page of the Solr admin UI shows all of the options to the JVM.
>> That will include the choice of garbage collector.
>>
>> You can also see the options with “ps -ef | grep solr”.
>>
>> wunder
>> Walter Underwood
>> [hidden email]
>> http://observer.wunderwood.org/  (my blog)
>>
>>> On Oct 13, 2020, at 1:19 PM, Ryan W <[hidden email]> wrote:
>>>
>>> I think I have it sorted. At this point I'm using GCG1, I take it,
>> because
>>> most recently I started Solr as a service...
>>>
>>> service solr start
>>>
>>> And that is running solr by way of /etc/init.d/solr because I don't have
>>> any systemd unit for solr, as explained here...
>>>
>> https://askubuntu.com/questions/903354/difference-between-systemctl-and-service-commands
>>>
>>> And I can see in the System V script for solr that /etc/default/
>> solr.in.sh
>>> is the relevant config file.
>>>
>>>
>>> On Tue, Oct 13, 2020 at 11:23 AM Ryan W <[hidden email]> wrote:
>>>
>>>> Or, perhaps if I start solr like so....
>>>>
>>>> service solr start
>>>>
>>>> ...it will use the solr.in.sh at /etc/default/solr.in.sh ?
>>>>
>>>>
>>>>
>>>> On Tue, Oct 13, 2020 at 11:19 AM Ryan W <[hidden email]> wrote:
>>>>
>>>>> This is how I start solr:
>>>>>
>>>>> /opt/solr/bin/solr start
>>>>>
>>>>> In my /etc/default/solr.in.sh, I have this...
>>>>>
>>>>> GC_TUNE=" \
>>>>> -XX:+UseG1GC \
>>>>> -XX:+ParallelRefProcEnabled \
>>>>> -XX:G1HeapRegionSize=8m \
>>>>> -XX:MaxGCPauseMillis=200 \
>>>>> -XX:+UseLargePages \
>>>>> -XX:+AggressiveOpts \
>>>>> "
>>>>>
>>>>> But I don't know how to tell if Solr is using that file.
>>>>>
>>>>> In my /opt/solr/bin there is no solr.in.sh, but there is a
>>>>> solr.in.sh.orig -- perhaps I should copy my /etc/default/solr.in.sh to
>>>>> /opt/solr/bin ?
>>>>>
>>>>> I am running Linux (RHEL).  The Solr version is 7.7.2.  Solr 8.x is not
>>>>> compatible with my application.
>>>>>
>>>>> Thank you.
>>>>>
>>>>>
>>>>> On Mon, Oct 12, 2020 at 9:46 PM Shawn Heisey <[hidden email]>
>>>>> wrote:
>>>>>
>>>>>> On 10/12/2020 5:11 PM, Ryan W wrote:
>>>>>>> Thanks.  How do I activate the G1GC collector?  Do I do this by
>>>>>> editing a
>>>>>>> config file, or by adding a parameter when I start solr?
>>>>>>>
>>>>>>> Oracle's docs are pointing me to a file that supposedly is at
>>>>>>> instance-dir/OUD/config/java.properties, but I don't have that path.
>>>>>> I am
>>>>>>> not sure what is meant by instance-dir here, but perhaps it means my
>>>>>> JRE
>>>>>>> install, which is at
>>>>>>> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.262.b10-0.el7_8.x86_64/jre --
>> but
>>>>>>> there is no "OUD" directory in this location.
>>>>>>
>>>>>> The collector is chosen by the startup options given to Java, in this
>>>>>> case by the start script for Solr.  I've never heard of it being set
>> by
>>>>>> a config in the JRE.
>>>>>>
>>>>>> In Solr 7, the start script defaults to the CMS collector.  We have
>>>>>> updated that to G1 in the latest Solr 8.x versions, because CMS has
>> been
>>>>>> deprecated by Oracle.
>>>>>>
>>>>>> Adding the following lines to the correct solr.in.sh would change the
>>>>>> garbage collector to G1.  I got this from the "bin/solr" script in
>> Solr
>>>>>> 8.5.1:
>>>>>>
>>>>>>      GC_TUNE=('-XX:+UseG1GC' \
>>>>>>        '-XX:+PerfDisableSharedMem' \
>>>>>>        '-XX:+ParallelRefProcEnabled' \
>>>>>>        '-XX:MaxGCPauseMillis=250' \
>>>>>>        '-XX:+UseLargePages' \
>>>>>>        '-XX:+AlwaysPreTouch')
>>>>>>
>>>>>> If you used the service installer script to install Solr, then the
>>>>>> correct file to add this to is usually /etc/default/solr.in.sh ...
>> but
>>>>>> if you did the install manually, it may be in the same bin directory
>>>>>> that contains the solr script itself.  Your initial message says the
>>>>>> solr home is /opt/solr/server/solr so I am assuming it's not running
>> on
>>>>>> Windows.
>>>>>>
>>>>>> Thanks,
>>>>>> Shawn
>>>>>>
>>>>>
>>
>>