Some performance questions....

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
64 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Some performance questions....

BlackIce
Hi,

I have some questions regarding performance.

Lets says I have a dual CPU with a total of 8 cores and 24 GB RAM for my
Solr and some other stuff.

Would it be more beneficial to only run 1 instance of Solr with the
collection stored on 4 HD's in RAID 0?? Or.... Have several Virtual
Machines each running of its own HD, ie: Have 4 VM's running Solr?

Any Thoughts?

Thank you!

RRK
Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

Deepak Goel
To rephrase your Question

"Does Solr do well with Scale-up or Scale-out?"

Are there any Performance Benchmarks for the same out there supporting the
claim?

On 11 Mar 2018 23:05, "BlackIce" <[hidden email]> wrote:

> Hi,
>
> I have some questions regarding performance.
>
> Lets says I have a dual CPU with a total of 8 cores and 24 GB RAM for my
> Solr and some other stuff.
>
> Would it be more beneficial to only run 1 instance of Solr with the
> collection stored on 4 HD's in RAID 0?? Or.... Have several Virtual
> Machines each running of its own HD, ie: Have 4 VM's running Solr?
>
> Any Thoughts?
>
> Thank you!
>
> RRK
>
Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

BlackIce
Thnx for the pointers.

I haven't given much thought to Solr, asides shemal.xml and solrconfig.xml
and I'm just diving into a bit more deeper stuff!

Greetz

RRK

On Sun, Mar 11, 2018 at 8:58 PM, Deepak Goel <[hidden email]> wrote:

> To rephrase your Question
>
> "Does Solr do well with Scale-up or Scale-out?"
>
> Are there any Performance Benchmarks for the same out there supporting the
> claim?
>
> On 11 Mar 2018 23:05, "BlackIce" <[hidden email]> wrote:
>
> > Hi,
> >
> > I have some questions regarding performance.
> >
> > Lets says I have a dual CPU with a total of 8 cores and 24 GB RAM for my
> > Solr and some other stuff.
> >
> > Would it be more beneficial to only run 1 instance of Solr with the
> > collection stored on 4 HD's in RAID 0?? Or.... Have several Virtual
> > Machines each running of its own HD, ie: Have 4 VM's running Solr?
> >
> > Any Thoughts?
> >
> > Thank you!
> >
> > RRK
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

BlackIce
Second to this wouldn't 4 Solr instances each with its own HD be fault
tolerant? vs. one solr instance with 4 HD's in RAID 0? Plus to his comes
the storage capacity, I need the capacity of those 4 drives... the more I
read.. the more questions....

On Sun, Mar 11, 2018 at 9:43 PM, BlackIce <[hidden email]> wrote:

> Thnx for the pointers.
>
> I haven't given much thought to Solr, asides shemal.xml and solrconfig.xml
> and I'm just diving into a bit more deeper stuff!
>
> Greetz
>
> RRK
>
> On Sun, Mar 11, 2018 at 8:58 PM, Deepak Goel <[hidden email]> wrote:
>
>> To rephrase your Question
>>
>> "Does Solr do well with Scale-up or Scale-out?"
>>
>> Are there any Performance Benchmarks for the same out there supporting the
>> claim?
>>
>> On 11 Mar 2018 23:05, "BlackIce" <[hidden email]> wrote:
>>
>> > Hi,
>> >
>> > I have some questions regarding performance.
>> >
>> > Lets says I have a dual CPU with a total of 8 cores and 24 GB RAM for my
>> > Solr and some other stuff.
>> >
>> > Would it be more beneficial to only run 1 instance of Solr with the
>> > collection stored on 4 HD's in RAID 0?? Or.... Have several Virtual
>> > Machines each running of its own HD, ie: Have 4 VM's running Solr?
>> >
>> > Any Thoughts?
>> >
>> > Thank you!
>> >
>> > RRK
>> >
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

Shawn Heisey-2
In reply to this post by BlackIce
On 3/11/2018 11:35 AM, BlackIce wrote:
> I have some questions regarding performance.
>
> Lets says I have a dual CPU with a total of 8 cores and 24 GB RAM for my
> Solr and some other stuff.
>
> Would it be more beneficial to only run 1 instance of Solr with the
> collection stored on 4 HD's in RAID 0?? Or.... Have several Virtual
> Machines each running of its own HD, ie: Have 4 VM's running Solr?

Performance is always going to be better on bare metal than on virtual
machines.  Virtualization in modern times is really good, so the
difference *might* be minimal, but there is ALWAYS overhead.

I used to create virtual machines in my hardware for Solr. Initially
with vmware esxi, then later natively in Linux with KVM.  At that time,
I was running one index core per VM.  Just for some testing, I took a
similar machine and set up one Solr instance handling all the same cores
on bare metal.  I do not remember HOW much faster it was, but it was
definitely faster. One big thing I like about bare metal is that there's
only one "machine", IP address, and Solr instance to administer.

Unless you're willing to completely rebuild the whole thing in the event
of drive failure, don't use RAID0.  If one drive dies (and every hard
drive IS eventually going to die if it's used long enough), then *all*
of the data on the whole RAID volume is gone.

You could do RAID5, which has decent redundancy and good space
efficiency, but if you're not familiar with the RAID5 write penalty, do
some research on it, and you'll probably come out of it not wanting to
EVER use it.  If you like, I can explain exactly why you should avoid
any RAID level that incorporates 5 or 6.

Overall, the best level is RAID10 ... but it has a glaring disadvantage
from a cost perspective -- you lose half of your raw capacity.  Since
drives are relatively cheap, I always build my servers with RAID10,
using a 1MB stripe size and a battery-backed caching controller.  For
the typical hardware I'm using, that means that I'm going to end up with
6 to 12TB of usable space instead of 10 to 20TB (raid5), but the volume
is FAST.

Thanks,
Shawn

Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

Deepak Goel
On 12 Mar 2018 05:51, "Shawn Heisey" <[hidden email]> wrote:

On 3/11/2018 11:35 AM, BlackIce wrote:

> I have some questions regarding performance.
>
> Lets says I have a dual CPU with a total of 8 cores and 24 GB RAM for my
> Solr and some other stuff.
>
> Would it be more beneficial to only run 1 instance of Solr with the
> collection stored on 4 HD's in RAID 0?? Or.... Have several Virtual
> Machines each running of its own HD, ie: Have 4 VM's running Solr?
>

Performance is always going to be better on bare metal than on virtual
machines.  Virtualization in modern times is really good, so the difference
*might* be minimal, but there is ALWAYS overhead.

*****Deepak*****

I doubt this. It would be great if someone can subtantiate this with hard
facts
*****Deepak*****


I used to create virtual machines in my hardware for Solr. Initially with
vmware esxi, then later natively in Linux with KVM.  At that time, I was
running one index core per VM.  Just for some testing, I took a similar
machine and set up one Solr instance handling all the same cores on bare
metal.  I do not remember HOW much faster it was, but it was definitely
faster. One big thing I like about bare metal is that there's only one
"machine", IP address, and Solr instance to administer.

Unless you're willing to completely rebuild the whole thing in the event of
drive failure, don't use RAID0.  If one drive dies (and every hard drive IS
eventually going to die if it's used long enough), then *all* of the data
on the whole RAID volume is gone.

You could do RAID5, which has decent redundancy and good space efficiency,
but if you're not familiar with the RAID5 write penalty, do some research
on it, and you'll probably come out of it not wanting to EVER use it.  If
you like, I can explain exactly why you should avoid any RAID level that
incorporates 5 or 6.

Overall, the best level is RAID10 ... but it has a glaring disadvantage
from a cost perspective -- you lose half of your raw capacity.  Since
drives are relatively cheap, I always build my servers with RAID10, using a
1MB stripe size and a battery-backed caching controller.  For the typical
hardware I'm using, that means that I'm going to end up with 6 to 12TB of
usable space instead of 10 to 20TB (raid5), but the volume is FAST.

Thanks,
Shawn
Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

Shawn Heisey-2
On 3/11/2018 7:39 PM, Deepak Goel wrote:
> I doubt this. It would be great if someone can subtantiate this with hard
> facts

This seems to be in response to my claim that virtualization always has
overhead.  I don't see how this statement can be at all controversial.

Virtualization isn't free, even if the hardware and software in use are
extremely efficient at it.  Translating what a virtual machine does into
a corresponding action on the real hardware is going to take time and
resources beyond whatever the action itself is.

Plus there's the application-level overhead.  You have the overhead of
multiple operating systems, multiple copies of Java running, multiple
servlet containers (probably Jetty), and multiple copies of Solr.  And
each of them is running inside a limited subset of the hardware
installed in the physical server.

Let's say you start with VMs on a server, and benchmark Solr's
performance.  Then you completely erase the server, install one
operating system, install Solr onto the OS, and then install all of the
indexes that were running on the VMs into that one Solr instance. 
Assuming that things are set up correctly and that you give that Solr
instance the correct amount of heap memory, it's almost guaranteed to be
faster than the VMs.  I can't tell you whether the improvement will be
half a percent or 50 percent, only that it will be faster.

Thanks,
Shawn

Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

Deepak Goel
We need benchmarks or data to support the claim.

A single OS and JVM does not scale linearly for higher loads. If you have
seperate OS and Java, the load is distributed across multiple instances
(with each instance only requiered to support a smaller load and hence
would scale nicely)

I had found this for running multiple apache servers on multiple VMs as
compared to a single instance (not Solr). But i am pretty sure it would be
same for Solr too

On 12 Mar 2018 12:42, "Shawn Heisey" <[hidden email]> wrote:

> On 3/11/2018 7:39 PM, Deepak Goel wrote:
>
>> I doubt this. It would be great if someone can subtantiate this with hard
>> facts
>>
>
> This seems to be in response to my claim that virtualization always has
> overhead.  I don't see how this statement can be at all controversial.
>
> Virtualization isn't free, even if the hardware and software in use are
> extremely efficient at it.  Translating what a virtual machine does into a
> corresponding action on the real hardware is going to take time and
> resources beyond whatever the action itself is.
>
> Plus there's the application-level overhead.  You have the overhead of
> multiple operating systems, multiple copies of Java running, multiple
> servlet containers (probably Jetty), and multiple copies of Solr.  And each
> of them is running inside a limited subset of the hardware installed in the
> physical server.
>
> Let's say you start with VMs on a server, and benchmark Solr's
> performance.  Then you completely erase the server, install one operating
> system, install Solr onto the OS, and then install all of the indexes that
> were running on the VMs into that one Solr instance.  Assuming that things
> are set up correctly and that you give that Solr instance the correct
> amount of heap memory, it's almost guaranteed to be faster than the VMs.  I
> can't tell you whether the improvement will be half a percent or 50
> percent, only that it will be faster.
>
> Thanks,
> Shawn
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

Deepak Goel
In reply to this post by BlackIce
Now you are mixing your original question about performance with reliability

On 12 Mar 2018 02:29, "BlackIce" <[hidden email]> wrote:

> Second to this wouldn't 4 Solr instances each with its own HD be fault
> tolerant? vs. one solr instance with 4 HD's in RAID 0? Plus to his comes
> the storage capacity, I need the capacity of those 4 drives... the more I
> read.. the more questions....
>
> On Sun, Mar 11, 2018 at 9:43 PM, BlackIce <[hidden email]> wrote:
>
> > Thnx for the pointers.
> >
> > I haven't given much thought to Solr, asides shemal.xml and
> solrconfig.xml
> > and I'm just diving into a bit more deeper stuff!
> >
> > Greetz
> >
> > RRK
> >
> > On Sun, Mar 11, 2018 at 8:58 PM, Deepak Goel <[hidden email]> wrote:
> >
> >> To rephrase your Question
> >>
> >> "Does Solr do well with Scale-up or Scale-out?"
> >>
> >> Are there any Performance Benchmarks for the same out there supporting
> the
> >> claim?
> >>
> >> On 11 Mar 2018 23:05, "BlackIce" <[hidden email]> wrote:
> >>
> >> > Hi,
> >> >
> >> > I have some questions regarding performance.
> >> >
> >> > Lets says I have a dual CPU with a total of 8 cores and 24 GB RAM for
> my
> >> > Solr and some other stuff.
> >> >
> >> > Would it be more beneficial to only run 1 instance of Solr with the
> >> > collection stored on 4 HD's in RAID 0?? Or.... Have several Virtual
> >> > Machines each running of its own HD, ie: Have 4 VM's running Solr?
> >> >
> >> > Any Thoughts?
> >> >
> >> > Thank you!
> >> >
> >> > RRK
> >> >
> >>
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

Shawn Heisey-2
In reply to this post by Deepak Goel
On 3/12/2018 3:22 AM, Deepak Goel wrote:
> A single OS and JVM does not scale linearly for higher loads. If you have
> seperate OS and Java, the load is distributed across multiple instances
> (with each instance only requiered to support a smaller load and hence
> would scale nicely)
>
> I had found this for running multiple apache servers on multiple VMs as
> compared to a single instance (not Solr). But i am pretty sure it would be
> same for Solr too

I think this is the last thing I'm going to say on the subject.  You
disagree with a fundamental hardware concept that I've learned through
experience, so I might never convince you of anything.  If that's the
case, I'm done trying after this, and I wish you the best of luck with
your efforts.

If the physical hosts you put the VMs on are far more powerful than you
would ever use for bare metal, and/or you split virtual machines between
different physical hosts, then that installation might scale better than
a single bare metal host. The decision makers in most companies are
usually a lot more willing to buy really expensive hardware if you tell
them it's for virtualization than they are for a single-purpose machine.

But if the bare metal environment has the same number of physical
servers with the same specifications, then a well-tuned bare metal setup
is going to perform better than virtual machines.  There's nothing wrong
with VMs.  They can perform very well if everything's sized appropriately.

Anytime a virtualized environment performs better than bare metal, it's
usually going to be that way because the virtualized environment has
different hardware than the bare metal environment.  That hardware will
probably be much more expensive, and/or newer hardware that just works
better.  It might also happen because the software installation was not
set up correctly to fully utilize all the hardware.

Solr works best with a lot more memory installed than people usually
install, *especially* with virtual machines, where RAM may be an even
more precious commodity than it is in bare metal servers.

Thanks,
Shawn

Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

BlackIce
In reply to this post by Deepak Goel
OK, so we're gone nowhere,  since I've already lost lots of time...  A few
days more or less won't make a difference....  I'd be willing to benchmark
if some tells me how to.


Greetz

On Mar 12, 2018 7:17 AM, "Deepak Goel" <[hidden email]> wrote:

> Now you are mixing your original question about performance with
> reliability
>
> On 12 Mar 2018 02:29, "BlackIce" <[hidden email]> wrote:
>
> > Second to this wouldn't 4 Solr instances each with its own HD be fault
> > tolerant? vs. one solr instance with 4 HD's in RAID 0? Plus to his comes
> > the storage capacity, I need the capacity of those 4 drives... the more I
> > read.. the more questions....
> >
> > On Sun, Mar 11, 2018 at 9:43 PM, BlackIce <[hidden email]> wrote:
> >
> > > Thnx for the pointers.
> > >
> > > I haven't given much thought to Solr, asides shemal.xml and
> > solrconfig.xml
> > > and I'm just diving into a bit more deeper stuff!
> > >
> > > Greetz
> > >
> > > RRK
> > >
> > > On Sun, Mar 11, 2018 at 8:58 PM, Deepak Goel <[hidden email]>
> wrote:
> > >
> > >> To rephrase your Question
> > >>
> > >> "Does Solr do well with Scale-up or Scale-out?"
> > >>
> > >> Are there any Performance Benchmarks for the same out there supporting
> > the
> > >> claim?
> > >>
> > >> On 11 Mar 2018 23:05, "BlackIce" <[hidden email]> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > I have some questions regarding performance.
> > >> >
> > >> > Lets says I have a dual CPU with a total of 8 cores and 24 GB RAM
> for
> > my
> > >> > Solr and some other stuff.
> > >> >
> > >> > Would it be more beneficial to only run 1 instance of Solr with the
> > >> > collection stored on 4 HD's in RAID 0?? Or.... Have several Virtual
> > >> > Machines each running of its own HD, ie: Have 4 VM's running Solr?
> > >> >
> > >> > Any Thoughts?
> > >> >
> > >> > Thank you!
> > >> >
> > >> > RRK
> > >> >
> > >>
> > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

Walter Underwood
Benchmark with production logs. Replay them at a constant request rate. Measure the response time and look at the median and 90th or 95th percentile. Do not use the average response time, because that will be thrown off by outliers.

It is best to run a few thousand warming queries before starting the measured benchmark run. That will load some results into the caches and will also load the index files into OS file buffers.

Restart the processes between benchmark runs to clear the caches.

My benchmark runs are about an hour long, not counting warming and analysis.

I always configure Solr with enough RAM that all the indexes fit into OS file buffers. With that, disk speed only matters for startup and indexing.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)

> On Mar 12, 2018, at 10:16 AM, BlackIce <[hidden email]> wrote:
>
> OK, so we're gone nowhere,  since I've already lost lots of time...  A few
> days more or less won't make a difference....  I'd be willing to benchmark
> if some tells me how to.
>
>
> Greetz
>
> On Mar 12, 2018 7:17 AM, "Deepak Goel" <[hidden email]> wrote:
>
>> Now you are mixing your original question about performance with
>> reliability
>>
>> On 12 Mar 2018 02:29, "BlackIce" <[hidden email]> wrote:
>>
>>> Second to this wouldn't 4 Solr instances each with its own HD be fault
>>> tolerant? vs. one solr instance with 4 HD's in RAID 0? Plus to his comes
>>> the storage capacity, I need the capacity of those 4 drives... the more I
>>> read.. the more questions....
>>>
>>> On Sun, Mar 11, 2018 at 9:43 PM, BlackIce <[hidden email]> wrote:
>>>
>>>> Thnx for the pointers.
>>>>
>>>> I haven't given much thought to Solr, asides shemal.xml and
>>> solrconfig.xml
>>>> and I'm just diving into a bit more deeper stuff!
>>>>
>>>> Greetz
>>>>
>>>> RRK
>>>>
>>>> On Sun, Mar 11, 2018 at 8:58 PM, Deepak Goel <[hidden email]>
>> wrote:
>>>>
>>>>> To rephrase your Question
>>>>>
>>>>> "Does Solr do well with Scale-up or Scale-out?"
>>>>>
>>>>> Are there any Performance Benchmarks for the same out there supporting
>>> the
>>>>> claim?
>>>>>
>>>>> On 11 Mar 2018 23:05, "BlackIce" <[hidden email]> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I have some questions regarding performance.
>>>>>>
>>>>>> Lets says I have a dual CPU with a total of 8 cores and 24 GB RAM
>> for
>>> my
>>>>>> Solr and some other stuff.
>>>>>>
>>>>>> Would it be more beneficial to only run 1 instance of Solr with the
>>>>>> collection stored on 4 HD's in RAID 0?? Or.... Have several Virtual
>>>>>> Machines each running of its own HD, ie: Have 4 VM's running Solr?
>>>>>>
>>>>>> Any Thoughts?
>>>>>>
>>>>>> Thank you!
>>>>>>
>>>>>> RRK
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

BlackIce
In reply to this post by BlackIce
So Im thinking following scenarios :
Single instance with drives in raid 0, raid 10 and raid 5.

And then having 3 Vms and 4 Solr instances each with its own HD.

How do I test this?


Greetz

On Mar 12, 2018 1:16 PM, "BlackIce" <[hidden email]> wrote:

> OK, so we're gone nowhere,  since I've already lost lots of time...  A few
> days more or less won't make a difference....  I'd be willing to benchmark
> if some tells me how to.
>
>
> Greetz
>
> On Mar 12, 2018 7:17 AM, "Deepak Goel" <[hidden email]> wrote:
>
>> Now you are mixing your original question about performance with
>> reliability
>>
>> On 12 Mar 2018 02:29, "BlackIce" <[hidden email]> wrote:
>>
>> > Second to this wouldn't 4 Solr instances each with its own HD be fault
>> > tolerant? vs. one solr instance with 4 HD's in RAID 0? Plus to his comes
>> > the storage capacity, I need the capacity of those 4 drives... the more
>> I
>> > read.. the more questions....
>> >
>> > On Sun, Mar 11, 2018 at 9:43 PM, BlackIce <[hidden email]>
>> wrote:
>> >
>> > > Thnx for the pointers.
>> > >
>> > > I haven't given much thought to Solr, asides shemal.xml and
>> > solrconfig.xml
>> > > and I'm just diving into a bit more deeper stuff!
>> > >
>> > > Greetz
>> > >
>> > > RRK
>> > >
>> > > On Sun, Mar 11, 2018 at 8:58 PM, Deepak Goel <[hidden email]>
>> wrote:
>> > >
>> > >> To rephrase your Question
>> > >>
>> > >> "Does Solr do well with Scale-up or Scale-out?"
>> > >>
>> > >> Are there any Performance Benchmarks for the same out there
>> supporting
>> > the
>> > >> claim?
>> > >>
>> > >> On 11 Mar 2018 23:05, "BlackIce" <[hidden email]> wrote:
>> > >>
>> > >> > Hi,
>> > >> >
>> > >> > I have some questions regarding performance.
>> > >> >
>> > >> > Lets says I have a dual CPU with a total of 8 cores and 24 GB RAM
>> for
>> > my
>> > >> > Solr and some other stuff.
>> > >> >
>> > >> > Would it be more beneficial to only run 1 instance of Solr with the
>> > >> > collection stored on 4 HD's in RAID 0?? Or.... Have several Virtual
>> > >> > Machines each running of its own HD, ie: Have 4 VM's running Solr?
>> > >> >
>> > >> > Any Thoughts?
>> > >> >
>> > >> > Thank you!
>> > >> >
>> > >> > RRK
>> > >> >
>> > >>
>> > >
>> > >
>> >
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

Deepak Goel
I am not sure if I understand your question

*"How do I test this?"*
You have to run test (benchmark test) of transactions (queries) which are
most representative of your system (requirement).

You can use a performance testing tool like JMeter (along with PerfMon
configured for utilisation metrics)



Deepak
"Please stop cruelty to Animals, help by becoming a Vegan"
+91 73500 12833
[hidden email]

Facebook: https://www.facebook.com/deicool
LinkedIn: www.linkedin.com/in/deicool

"Plant a Tree, Go Green"

On Mon, Mar 12, 2018 at 10:57 PM, BlackIce <[hidden email]> wrote:

> So Im thinking following scenarios :
> Single instance with drives in raid 0, raid 10 and raid 5.
>
> And then having 3 Vms and 4 Solr instances each with its own HD.
>
> How do I test this?
>
>
> Greetz
>
> On Mar 12, 2018 1:16 PM, "BlackIce" <[hidden email]> wrote:
>
> > OK, so we're gone nowhere,  since I've already lost lots of time...  A
> few
> > days more or less won't make a difference....  I'd be willing to
> benchmark
> > if some tells me how to.
> >
> >
> > Greetz
> >
> > On Mar 12, 2018 7:17 AM, "Deepak Goel" <[hidden email]> wrote:
> >
> >> Now you are mixing your original question about performance with
> >> reliability
> >>
> >> On 12 Mar 2018 02:29, "BlackIce" <[hidden email]> wrote:
> >>
> >> > Second to this wouldn't 4 Solr instances each with its own HD be fault
> >> > tolerant? vs. one solr instance with 4 HD's in RAID 0? Plus to his
> comes
> >> > the storage capacity, I need the capacity of those 4 drives... the
> more
> >> I
> >> > read.. the more questions....
> >> >
> >> > On Sun, Mar 11, 2018 at 9:43 PM, BlackIce <[hidden email]>
> >> wrote:
> >> >
> >> > > Thnx for the pointers.
> >> > >
> >> > > I haven't given much thought to Solr, asides shemal.xml and
> >> > solrconfig.xml
> >> > > and I'm just diving into a bit more deeper stuff!
> >> > >
> >> > > Greetz
> >> > >
> >> > > RRK
> >> > >
> >> > > On Sun, Mar 11, 2018 at 8:58 PM, Deepak Goel <[hidden email]>
> >> wrote:
> >> > >
> >> > >> To rephrase your Question
> >> > >>
> >> > >> "Does Solr do well with Scale-up or Scale-out?"
> >> > >>
> >> > >> Are there any Performance Benchmarks for the same out there
> >> supporting
> >> > the
> >> > >> claim?
> >> > >>
> >> > >> On 11 Mar 2018 23:05, "BlackIce" <[hidden email]> wrote:
> >> > >>
> >> > >> > Hi,
> >> > >> >
> >> > >> > I have some questions regarding performance.
> >> > >> >
> >> > >> > Lets says I have a dual CPU with a total of 8 cores and 24 GB RAM
> >> for
> >> > my
> >> > >> > Solr and some other stuff.
> >> > >> >
> >> > >> > Would it be more beneficial to only run 1 instance of Solr with
> the
> >> > >> > collection stored on 4 HD's in RAID 0?? Or.... Have several
> Virtual
> >> > >> > Machines each running of its own HD, ie: Have 4 VM's running
> Solr?
> >> > >> >
> >> > >> > Any Thoughts?
> >> > >> >
> >> > >> > Thank you!
> >> > >> >
> >> > >> > RRK
> >> > >> >
> >> > >>
> >> > >
> >> > >
> >> >
> >>
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

BlackIce
I don't have any production logs.... and this all sounds to complicated.

So, I'll just trow the system together in a way it makes the most sense for
now.. collect some logs and then do some testing further down the road. For
now just get the sucker up and running.

Thanks all

On Mon, Mar 12, 2018 at 7:23 PM, Deepak Goel <[hidden email]> wrote:

> I am not sure if I understand your question
>
> *"How do I test this?"*
> You have to run test (benchmark test) of transactions (queries) which are
> most representative of your system (requirement).
>
> You can use a performance testing tool like JMeter (along with PerfMon
> configured for utilisation metrics)
>
>
>
> Deepak
> "Please stop cruelty to Animals, help by becoming a Vegan"
> +91 73500 12833
> [hidden email]
>
> Facebook: https://www.facebook.com/deicool
> LinkedIn: www.linkedin.com/in/deicool
>
> "Plant a Tree, Go Green"
>
> On Mon, Mar 12, 2018 at 10:57 PM, BlackIce <[hidden email]> wrote:
>
> > So Im thinking following scenarios :
> > Single instance with drives in raid 0, raid 10 and raid 5.
> >
> > And then having 3 Vms and 4 Solr instances each with its own HD.
> >
> > How do I test this?
> >
> >
> > Greetz
> >
> > On Mar 12, 2018 1:16 PM, "BlackIce" <[hidden email]> wrote:
> >
> > > OK, so we're gone nowhere,  since I've already lost lots of time...  A
> > few
> > > days more or less won't make a difference....  I'd be willing to
> > benchmark
> > > if some tells me how to.
> > >
> > >
> > > Greetz
> > >
> > > On Mar 12, 2018 7:17 AM, "Deepak Goel" <[hidden email]> wrote:
> > >
> > >> Now you are mixing your original question about performance with
> > >> reliability
> > >>
> > >> On 12 Mar 2018 02:29, "BlackIce" <[hidden email]> wrote:
> > >>
> > >> > Second to this wouldn't 4 Solr instances each with its own HD be
> fault
> > >> > tolerant? vs. one solr instance with 4 HD's in RAID 0? Plus to his
> > comes
> > >> > the storage capacity, I need the capacity of those 4 drives... the
> > more
> > >> I
> > >> > read.. the more questions....
> > >> >
> > >> > On Sun, Mar 11, 2018 at 9:43 PM, BlackIce <[hidden email]>
> > >> wrote:
> > >> >
> > >> > > Thnx for the pointers.
> > >> > >
> > >> > > I haven't given much thought to Solr, asides shemal.xml and
> > >> > solrconfig.xml
> > >> > > and I'm just diving into a bit more deeper stuff!
> > >> > >
> > >> > > Greetz
> > >> > >
> > >> > > RRK
> > >> > >
> > >> > > On Sun, Mar 11, 2018 at 8:58 PM, Deepak Goel <[hidden email]>
> > >> wrote:
> > >> > >
> > >> > >> To rephrase your Question
> > >> > >>
> > >> > >> "Does Solr do well with Scale-up or Scale-out?"
> > >> > >>
> > >> > >> Are there any Performance Benchmarks for the same out there
> > >> supporting
> > >> > the
> > >> > >> claim?
> > >> > >>
> > >> > >> On 11 Mar 2018 23:05, "BlackIce" <[hidden email]> wrote:
> > >> > >>
> > >> > >> > Hi,
> > >> > >> >
> > >> > >> > I have some questions regarding performance.
> > >> > >> >
> > >> > >> > Lets says I have a dual CPU with a total of 8 cores and 24 GB
> RAM
> > >> for
> > >> > my
> > >> > >> > Solr and some other stuff.
> > >> > >> >
> > >> > >> > Would it be more beneficial to only run 1 instance of Solr with
> > the
> > >> > >> > collection stored on 4 HD's in RAID 0?? Or.... Have several
> > Virtual
> > >> > >> > Machines each running of its own HD, ie: Have 4 VM's running
> > Solr?
> > >> > >> >
> > >> > >> > Any Thoughts?
> > >> > >> >
> > >> > >> > Thank you!
> > >> > >> >
> > >> > >> > RRK
> > >> > >> >
> > >> > >>
> > >> > >
> > >> > >
> > >> >
> > >>
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

BlackIce
I was just thinking.... Do I really need separate VM's in order to run
multiple Solr instances? Doesn't it suffice to have each instance in its
own user account?

Greetz

On Mon, Mar 12, 2018 at 7:41 PM, BlackIce <[hidden email]> wrote:

> I don't have any production logs.... and this all sounds to complicated.
>
> So, I'll just trow the system together in a way it makes the most sense
> for now.. collect some logs and then do some testing further down the road.
> For now just get the sucker up and running.
>
> Thanks all
>
> On Mon, Mar 12, 2018 at 7:23 PM, Deepak Goel <[hidden email]> wrote:
>
>> I am not sure if I understand your question
>>
>> *"How do I test this?"*
>> You have to run test (benchmark test) of transactions (queries) which are
>> most representative of your system (requirement).
>>
>> You can use a performance testing tool like JMeter (along with PerfMon
>> configured for utilisation metrics)
>>
>>
>>
>> Deepak
>> "Please stop cruelty to Animals, help by becoming a Vegan"
>> +91 73500 12833
>> [hidden email]
>>
>> Facebook: https://www.facebook.com/deicool
>> LinkedIn: www.linkedin.com/in/deicool
>>
>> "Plant a Tree, Go Green"
>>
>> On Mon, Mar 12, 2018 at 10:57 PM, BlackIce <[hidden email]> wrote:
>>
>> > So Im thinking following scenarios :
>> > Single instance with drives in raid 0, raid 10 and raid 5.
>> >
>> > And then having 3 Vms and 4 Solr instances each with its own HD.
>> >
>> > How do I test this?
>> >
>> >
>> > Greetz
>> >
>> > On Mar 12, 2018 1:16 PM, "BlackIce" <[hidden email]> wrote:
>> >
>> > > OK, so we're gone nowhere,  since I've already lost lots of time...  A
>> > few
>> > > days more or less won't make a difference....  I'd be willing to
>> > benchmark
>> > > if some tells me how to.
>> > >
>> > >
>> > > Greetz
>> > >
>> > > On Mar 12, 2018 7:17 AM, "Deepak Goel" <[hidden email]> wrote:
>> > >
>> > >> Now you are mixing your original question about performance with
>> > >> reliability
>> > >>
>> > >> On 12 Mar 2018 02:29, "BlackIce" <[hidden email]> wrote:
>> > >>
>> > >> > Second to this wouldn't 4 Solr instances each with its own HD be
>> fault
>> > >> > tolerant? vs. one solr instance with 4 HD's in RAID 0? Plus to his
>> > comes
>> > >> > the storage capacity, I need the capacity of those 4 drives... the
>> > more
>> > >> I
>> > >> > read.. the more questions....
>> > >> >
>> > >> > On Sun, Mar 11, 2018 at 9:43 PM, BlackIce <[hidden email]>
>> > >> wrote:
>> > >> >
>> > >> > > Thnx for the pointers.
>> > >> > >
>> > >> > > I haven't given much thought to Solr, asides shemal.xml and
>> > >> > solrconfig.xml
>> > >> > > and I'm just diving into a bit more deeper stuff!
>> > >> > >
>> > >> > > Greetz
>> > >> > >
>> > >> > > RRK
>> > >> > >
>> > >> > > On Sun, Mar 11, 2018 at 8:58 PM, Deepak Goel <[hidden email]>
>> > >> wrote:
>> > >> > >
>> > >> > >> To rephrase your Question
>> > >> > >>
>> > >> > >> "Does Solr do well with Scale-up or Scale-out?"
>> > >> > >>
>> > >> > >> Are there any Performance Benchmarks for the same out there
>> > >> supporting
>> > >> > the
>> > >> > >> claim?
>> > >> > >>
>> > >> > >> On 11 Mar 2018 23:05, "BlackIce" <[hidden email]> wrote:
>> > >> > >>
>> > >> > >> > Hi,
>> > >> > >> >
>> > >> > >> > I have some questions regarding performance.
>> > >> > >> >
>> > >> > >> > Lets says I have a dual CPU with a total of 8 cores and 24 GB
>> RAM
>> > >> for
>> > >> > my
>> > >> > >> > Solr and some other stuff.
>> > >> > >> >
>> > >> > >> > Would it be more beneficial to only run 1 instance of Solr
>> with
>> > the
>> > >> > >> > collection stored on 4 HD's in RAID 0?? Or.... Have several
>> > Virtual
>> > >> > >> > Machines each running of its own HD, ie: Have 4 VM's running
>> > Solr?
>> > >> > >> >
>> > >> > >> > Any Thoughts?
>> > >> > >> >
>> > >> > >> > Thank you!
>> > >> > >> >
>> > >> > >> > RRK
>> > >> > >> >
>> > >> > >>
>> > >> > >
>> > >> > >
>> > >> >
>> > >>
>> > >
>> >
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

Deepak Goel
The OS resources would be shared in that case

On 14 Mar 2018 17:19, "BlackIce" <[hidden email]> wrote:

> I was just thinking.... Do I really need separate VM's in order to run
> multiple Solr instances? Doesn't it suffice to have each instance in its
> own user account?
>
> Greetz
>
> On Mon, Mar 12, 2018 at 7:41 PM, BlackIce <[hidden email]> wrote:
>
> > I don't have any production logs.... and this all sounds to complicated.
> >
> > So, I'll just trow the system together in a way it makes the most sense
> > for now.. collect some logs and then do some testing further down the
> road.
> > For now just get the sucker up and running.
> >
> > Thanks all
> >
> > On Mon, Mar 12, 2018 at 7:23 PM, Deepak Goel <[hidden email]> wrote:
> >
> >> I am not sure if I understand your question
> >>
> >> *"How do I test this?"*
> >> You have to run test (benchmark test) of transactions (queries) which
> are
> >> most representative of your system (requirement).
> >>
> >> You can use a performance testing tool like JMeter (along with PerfMon
> >> configured for utilisation metrics)
> >>
> >>
> >>
> >> Deepak
> >> "Please stop cruelty to Animals, help by becoming a Vegan"
> >> +91 73500 12833
> >> [hidden email]
> >>
> >> Facebook: https://www.facebook.com/deicool
> >> LinkedIn: www.linkedin.com/in/deicool
> >>
> >> "Plant a Tree, Go Green"
> >>
> >> On Mon, Mar 12, 2018 at 10:57 PM, BlackIce <[hidden email]>
> wrote:
> >>
> >> > So Im thinking following scenarios :
> >> > Single instance with drives in raid 0, raid 10 and raid 5.
> >> >
> >> > And then having 3 Vms and 4 Solr instances each with its own HD.
> >> >
> >> > How do I test this?
> >> >
> >> >
> >> > Greetz
> >> >
> >> > On Mar 12, 2018 1:16 PM, "BlackIce" <[hidden email]> wrote:
> >> >
> >> > > OK, so we're gone nowhere,  since I've already lost lots of
> time...  A
> >> > few
> >> > > days more or less won't make a difference....  I'd be willing to
> >> > benchmark
> >> > > if some tells me how to.
> >> > >
> >> > >
> >> > > Greetz
> >> > >
> >> > > On Mar 12, 2018 7:17 AM, "Deepak Goel" <[hidden email]> wrote:
> >> > >
> >> > >> Now you are mixing your original question about performance with
> >> > >> reliability
> >> > >>
> >> > >> On 12 Mar 2018 02:29, "BlackIce" <[hidden email]> wrote:
> >> > >>
> >> > >> > Second to this wouldn't 4 Solr instances each with its own HD be
> >> fault
> >> > >> > tolerant? vs. one solr instance with 4 HD's in RAID 0? Plus to
> his
> >> > comes
> >> > >> > the storage capacity, I need the capacity of those 4 drives...
> the
> >> > more
> >> > >> I
> >> > >> > read.. the more questions....
> >> > >> >
> >> > >> > On Sun, Mar 11, 2018 at 9:43 PM, BlackIce <[hidden email]
> >
> >> > >> wrote:
> >> > >> >
> >> > >> > > Thnx for the pointers.
> >> > >> > >
> >> > >> > > I haven't given much thought to Solr, asides shemal.xml and
> >> > >> > solrconfig.xml
> >> > >> > > and I'm just diving into a bit more deeper stuff!
> >> > >> > >
> >> > >> > > Greetz
> >> > >> > >
> >> > >> > > RRK
> >> > >> > >
> >> > >> > > On Sun, Mar 11, 2018 at 8:58 PM, Deepak Goel <
> [hidden email]>
> >> > >> wrote:
> >> > >> > >
> >> > >> > >> To rephrase your Question
> >> > >> > >>
> >> > >> > >> "Does Solr do well with Scale-up or Scale-out?"
> >> > >> > >>
> >> > >> > >> Are there any Performance Benchmarks for the same out there
> >> > >> supporting
> >> > >> > the
> >> > >> > >> claim?
> >> > >> > >>
> >> > >> > >> On 11 Mar 2018 23:05, "BlackIce" <[hidden email]>
> wrote:
> >> > >> > >>
> >> > >> > >> > Hi,
> >> > >> > >> >
> >> > >> > >> > I have some questions regarding performance.
> >> > >> > >> >
> >> > >> > >> > Lets says I have a dual CPU with a total of 8 cores and 24
> GB
> >> RAM
> >> > >> for
> >> > >> > my
> >> > >> > >> > Solr and some other stuff.
> >> > >> > >> >
> >> > >> > >> > Would it be more beneficial to only run 1 instance of Solr
> >> with
> >> > the
> >> > >> > >> > collection stored on 4 HD's in RAID 0?? Or.... Have several
> >> > Virtual
> >> > >> > >> > Machines each running of its own HD, ie: Have 4 VM's running
> >> > Solr?
> >> > >> > >> >
> >> > >> > >> > Any Thoughts?
> >> > >> > >> >
> >> > >> > >> > Thank you!
> >> > >> > >> >
> >> > >> > >> > RRK
> >> > >> > >> >
> >> > >> > >>
> >> > >> > >
> >> > >> > >
> >> > >> >
> >> > >>
> >> > >
> >> >
> >>
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

BlackIce
but it should be possible, without the overhead of VM's

On Wed, Mar 14, 2018 at 1:30 PM, Deepak Goel <[hidden email]> wrote:

> The OS resources would be shared in that case
>
> On 14 Mar 2018 17:19, "BlackIce" <[hidden email]> wrote:
>
> > I was just thinking.... Do I really need separate VM's in order to run
> > multiple Solr instances? Doesn't it suffice to have each instance in its
> > own user account?
> >
> > Greetz
> >
> > On Mon, Mar 12, 2018 at 7:41 PM, BlackIce <[hidden email]> wrote:
> >
> > > I don't have any production logs.... and this all sounds to
> complicated.
> > >
> > > So, I'll just trow the system together in a way it makes the most sense
> > > for now.. collect some logs and then do some testing further down the
> > road.
> > > For now just get the sucker up and running.
> > >
> > > Thanks all
> > >
> > > On Mon, Mar 12, 2018 at 7:23 PM, Deepak Goel <[hidden email]>
> wrote:
> > >
> > >> I am not sure if I understand your question
> > >>
> > >> *"How do I test this?"*
> > >> You have to run test (benchmark test) of transactions (queries) which
> > are
> > >> most representative of your system (requirement).
> > >>
> > >> You can use a performance testing tool like JMeter (along with PerfMon
> > >> configured for utilisation metrics)
> > >>
> > >>
> > >>
> > >> Deepak
> > >> "Please stop cruelty to Animals, help by becoming a Vegan"
> > >> +91 73500 12833
> > >> [hidden email]
> > >>
> > >> Facebook: https://www.facebook.com/deicool
> > >> LinkedIn: www.linkedin.com/in/deicool
> > >>
> > >> "Plant a Tree, Go Green"
> > >>
> > >> On Mon, Mar 12, 2018 at 10:57 PM, BlackIce <[hidden email]>
> > wrote:
> > >>
> > >> > So Im thinking following scenarios :
> > >> > Single instance with drives in raid 0, raid 10 and raid 5.
> > >> >
> > >> > And then having 3 Vms and 4 Solr instances each with its own HD.
> > >> >
> > >> > How do I test this?
> > >> >
> > >> >
> > >> > Greetz
> > >> >
> > >> > On Mar 12, 2018 1:16 PM, "BlackIce" <[hidden email]> wrote:
> > >> >
> > >> > > OK, so we're gone nowhere,  since I've already lost lots of
> > time...  A
> > >> > few
> > >> > > days more or less won't make a difference....  I'd be willing to
> > >> > benchmark
> > >> > > if some tells me how to.
> > >> > >
> > >> > >
> > >> > > Greetz
> > >> > >
> > >> > > On Mar 12, 2018 7:17 AM, "Deepak Goel" <[hidden email]> wrote:
> > >> > >
> > >> > >> Now you are mixing your original question about performance with
> > >> > >> reliability
> > >> > >>
> > >> > >> On 12 Mar 2018 02:29, "BlackIce" <[hidden email]> wrote:
> > >> > >>
> > >> > >> > Second to this wouldn't 4 Solr instances each with its own HD
> be
> > >> fault
> > >> > >> > tolerant? vs. one solr instance with 4 HD's in RAID 0? Plus to
> > his
> > >> > comes
> > >> > >> > the storage capacity, I need the capacity of those 4 drives...
> > the
> > >> > more
> > >> > >> I
> > >> > >> > read.. the more questions....
> > >> > >> >
> > >> > >> > On Sun, Mar 11, 2018 at 9:43 PM, BlackIce <
> [hidden email]
> > >
> > >> > >> wrote:
> > >> > >> >
> > >> > >> > > Thnx for the pointers.
> > >> > >> > >
> > >> > >> > > I haven't given much thought to Solr, asides shemal.xml and
> > >> > >> > solrconfig.xml
> > >> > >> > > and I'm just diving into a bit more deeper stuff!
> > >> > >> > >
> > >> > >> > > Greetz
> > >> > >> > >
> > >> > >> > > RRK
> > >> > >> > >
> > >> > >> > > On Sun, Mar 11, 2018 at 8:58 PM, Deepak Goel <
> > [hidden email]>
> > >> > >> wrote:
> > >> > >> > >
> > >> > >> > >> To rephrase your Question
> > >> > >> > >>
> > >> > >> > >> "Does Solr do well with Scale-up or Scale-out?"
> > >> > >> > >>
> > >> > >> > >> Are there any Performance Benchmarks for the same out there
> > >> > >> supporting
> > >> > >> > the
> > >> > >> > >> claim?
> > >> > >> > >>
> > >> > >> > >> On 11 Mar 2018 23:05, "BlackIce" <[hidden email]>
> > wrote:
> > >> > >> > >>
> > >> > >> > >> > Hi,
> > >> > >> > >> >
> > >> > >> > >> > I have some questions regarding performance.
> > >> > >> > >> >
> > >> > >> > >> > Lets says I have a dual CPU with a total of 8 cores and 24
> > GB
> > >> RAM
> > >> > >> for
> > >> > >> > my
> > >> > >> > >> > Solr and some other stuff.
> > >> > >> > >> >
> > >> > >> > >> > Would it be more beneficial to only run 1 instance of Solr
> > >> with
> > >> > the
> > >> > >> > >> > collection stored on 4 HD's in RAID 0?? Or.... Have
> several
> > >> > Virtual
> > >> > >> > >> > Machines each running of its own HD, ie: Have 4 VM's
> running
> > >> > Solr?
> > >> > >> > >> >
> > >> > >> > >> > Any Thoughts?
> > >> > >> > >> >
> > >> > >> > >> > Thank you!
> > >> > >> > >> >
> > >> > >> > >> > RRK
> > >> > >> > >> >
> > >> > >> > >>
> > >> > >> > >
> > >> > >> > >
> > >> > >> >
> > >> > >>
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

Deepak Goel
Have you measured the overhead of VM anytime? Or have you read it somewhere?

On 14 Mar 2018 18:10, "BlackIce" <[hidden email]> wrote:

> but it should be possible, without the overhead of VM's
>
> On Wed, Mar 14, 2018 at 1:30 PM, Deepak Goel <[hidden email]> wrote:
>
> > The OS resources would be shared in that case
> >
> > On 14 Mar 2018 17:19, "BlackIce" <[hidden email]> wrote:
> >
> > > I was just thinking.... Do I really need separate VM's in order to run
> > > multiple Solr instances? Doesn't it suffice to have each instance in
> its
> > > own user account?
> > >
> > > Greetz
> > >
> > > On Mon, Mar 12, 2018 at 7:41 PM, BlackIce <[hidden email]>
> wrote:
> > >
> > > > I don't have any production logs.... and this all sounds to
> > complicated.
> > > >
> > > > So, I'll just trow the system together in a way it makes the most
> sense
> > > > for now.. collect some logs and then do some testing further down the
> > > road.
> > > > For now just get the sucker up and running.
> > > >
> > > > Thanks all
> > > >
> > > > On Mon, Mar 12, 2018 at 7:23 PM, Deepak Goel <[hidden email]>
> > wrote:
> > > >
> > > >> I am not sure if I understand your question
> > > >>
> > > >> *"How do I test this?"*
> > > >> You have to run test (benchmark test) of transactions (queries)
> which
> > > are
> > > >> most representative of your system (requirement).
> > > >>
> > > >> You can use a performance testing tool like JMeter (along with
> PerfMon
> > > >> configured for utilisation metrics)
> > > >>
> > > >>
> > > >>
> > > >> Deepak
> > > >> "Please stop cruelty to Animals, help by becoming a Vegan"
> > > >> +91 73500 12833
> > > >> [hidden email]
> > > >>
> > > >> Facebook: https://www.facebook.com/deicool
> > > >> LinkedIn: www.linkedin.com/in/deicool
> > > >>
> > > >> "Plant a Tree, Go Green"
> > > >>
> > > >> On Mon, Mar 12, 2018 at 10:57 PM, BlackIce <[hidden email]>
> > > wrote:
> > > >>
> > > >> > So Im thinking following scenarios :
> > > >> > Single instance with drives in raid 0, raid 10 and raid 5.
> > > >> >
> > > >> > And then having 3 Vms and 4 Solr instances each with its own HD.
> > > >> >
> > > >> > How do I test this?
> > > >> >
> > > >> >
> > > >> > Greetz
> > > >> >
> > > >> > On Mar 12, 2018 1:16 PM, "BlackIce" <[hidden email]>
> wrote:
> > > >> >
> > > >> > > OK, so we're gone nowhere,  since I've already lost lots of
> > > time...  A
> > > >> > few
> > > >> > > days more or less won't make a difference....  I'd be willing to
> > > >> > benchmark
> > > >> > > if some tells me how to.
> > > >> > >
> > > >> > >
> > > >> > > Greetz
> > > >> > >
> > > >> > > On Mar 12, 2018 7:17 AM, "Deepak Goel" <[hidden email]>
> wrote:
> > > >> > >
> > > >> > >> Now you are mixing your original question about performance
> with
> > > >> > >> reliability
> > > >> > >>
> > > >> > >> On 12 Mar 2018 02:29, "BlackIce" <[hidden email]>
> wrote:
> > > >> > >>
> > > >> > >> > Second to this wouldn't 4 Solr instances each with its own HD
> > be
> > > >> fault
> > > >> > >> > tolerant? vs. one solr instance with 4 HD's in RAID 0? Plus
> to
> > > his
> > > >> > comes
> > > >> > >> > the storage capacity, I need the capacity of those 4
> drives...
> > > the
> > > >> > more
> > > >> > >> I
> > > >> > >> > read.. the more questions....
> > > >> > >> >
> > > >> > >> > On Sun, Mar 11, 2018 at 9:43 PM, BlackIce <
> > [hidden email]
> > > >
> > > >> > >> wrote:
> > > >> > >> >
> > > >> > >> > > Thnx for the pointers.
> > > >> > >> > >
> > > >> > >> > > I haven't given much thought to Solr, asides shemal.xml and
> > > >> > >> > solrconfig.xml
> > > >> > >> > > and I'm just diving into a bit more deeper stuff!
> > > >> > >> > >
> > > >> > >> > > Greetz
> > > >> > >> > >
> > > >> > >> > > RRK
> > > >> > >> > >
> > > >> > >> > > On Sun, Mar 11, 2018 at 8:58 PM, Deepak Goel <
> > > [hidden email]>
> > > >> > >> wrote:
> > > >> > >> > >
> > > >> > >> > >> To rephrase your Question
> > > >> > >> > >>
> > > >> > >> > >> "Does Solr do well with Scale-up or Scale-out?"
> > > >> > >> > >>
> > > >> > >> > >> Are there any Performance Benchmarks for the same out
> there
> > > >> > >> supporting
> > > >> > >> > the
> > > >> > >> > >> claim?
> > > >> > >> > >>
> > > >> > >> > >> On 11 Mar 2018 23:05, "BlackIce" <[hidden email]>
> > > wrote:
> > > >> > >> > >>
> > > >> > >> > >> > Hi,
> > > >> > >> > >> >
> > > >> > >> > >> > I have some questions regarding performance.
> > > >> > >> > >> >
> > > >> > >> > >> > Lets says I have a dual CPU with a total of 8 cores and
> 24
> > > GB
> > > >> RAM
> > > >> > >> for
> > > >> > >> > my
> > > >> > >> > >> > Solr and some other stuff.
> > > >> > >> > >> >
> > > >> > >> > >> > Would it be more beneficial to only run 1 instance of
> Solr
> > > >> with
> > > >> > the
> > > >> > >> > >> > collection stored on 4 HD's in RAID 0?? Or.... Have
> > several
> > > >> > Virtual
> > > >> > >> > >> > Machines each running of its own HD, ie: Have 4 VM's
> > running
> > > >> > Solr?
> > > >> > >> > >> >
> > > >> > >> > >> > Any Thoughts?
> > > >> > >> > >> >
> > > >> > >> > >> > Thank you!
> > > >> > >> > >> >
> > > >> > >> > >> > RRK
> > > >> > >> > >> >
> > > >> > >> > >>
> > > >> > >> > >
> > > >> > >> > >
> > > >> > >> >
> > > >> > >>
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Some performance questions....

Shawn Heisey-2
In reply to this post by BlackIce
On 3/14/2018 5:49 AM, BlackIce wrote:
> I was just thinking.... Do I really need separate VM's in order to run
> multiple Solr instances? Doesn't it suffice to have each instance in its
> own user account?

You can run multiple instances all under the same account on one
machine.  But for a single machine, why do you need multiple Solr
instances at all?  One instance can handle many indexes, and will
probably do it more efficiently than multiple instances.

The only time I would *ever* recommend multiple Solr instances is when a
single instance would need an ENORMOUS Java heap -- something much
larger than 32GB.  If something like that can be split into multiple
instances where each one has a heap that's 31GB heap or less, then
memory usage will be more efficient and Java's garbage collection will
work better.

FYI -- Running Java with a 32GB heap actually has LESS memory available
than running it with a 31GB heap.  This is because when the heap reaches
32GB, Java must switch to 64-bit pointers, so every little allocation
requires a little bit more memory.

Thanks,
Shawn

1234