[Benchmark SOLR] JETTY VS TOMCAT

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[Benchmark SOLR] JETTY VS TOMCAT

Gerald Reinhart
Hello,

    We are migrating our platform
    from
         - Solr 5.4.1 hosted by a Tomcat
    to
         - Solr 5.4.1 standalone (hosted by Jetty)

=> Jetty is 15% slower than Tomcat in the same conditions.


   Here are details about the benchmarks :

   Context :
        - Index with 9 000 000 documents
        - Gatling launch queries extracted from the real traffic
        - Server :  R410 with 16 virtual CPU and 96G mem

   Results with 20 clients in // during 10 minutes:
    For Tomcat :
        - 165 Queries per seconds
        - 120ms mean response time

    For Jetty :
        - 139 Queries per seconds
        - 142ms mean response time

We have checked :
       - the load of the server => same
       - the io wait => same
       - the memory used in the JVM => same
       - JVM GC settings => same

      For us, it's a blocker for the migration.

      Is it a known issue ? (I found that :
http://www.asjava.com/jetty/jetty-vs-tomcat-performance-comparison/)

      How can we improve the performance of Jetty ? (We have already
followed
http://www.eclipse.org/jetty/documentation/9.2.21.v20170120/optimizing.html
recommendation)

     Many thanks,


Gérald Reinhart


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce message, merci de le détruire et d'en avertir l'expéditeur.
Reply | Threaded
Open this post in threaded view
|

Re: [Benchmark SOLR] JETTY VS TOMCAT

Billnbell
Did you try:

Set your acceptor count, SelectChannelConnector.setAcceptors(int)
<http://download.eclipse.org/jetty/stable-7/apidocs/org/eclipse/jetty/server/AbstractConnector.html#setAcceptors%28int%29>
to
be a a value between 1 and (number_of_cpu_cores - 1).

On Fri, Jan 27, 2017 at 3:22 AM, Gerald Reinhart <[hidden email]
> wrote:

> Hello,
>
>    We are migrating our platform
>    from
>         - Solr 5.4.1 hosted by a Tomcat
>    to
>         - Solr 5.4.1 standalone (hosted by Jetty)
>
> => Jetty is 15% slower than Tomcat in the same conditions.
>
>
>   Here are details about the benchmarks :
>
>   Context :
>        - Index with 9 000 000 documents
>        - Gatling launch queries extracted from the real traffic
>        - Server :  R410 with 16 virtual CPU and 96G mem
>
>   Results with 20 clients in // during 10 minutes:
>    For Tomcat :
>        - 165 Queries per seconds
>        - 120ms mean response time
>
>    For Jetty :
>        - 139 Queries per seconds
>        - 142ms mean response time
>
> We have checked :
>       - the load of the server => same
>       - the io wait => same
>       - the memory used in the JVM => same
>       - JVM GC settings => same
>
>      For us, it's a blocker for the migration.
>
>      Is it a known issue ? (I found that :
> http://www.asjava.com/jetty/jetty-vs-tomcat-performance-comparison/)
>
>      How can we improve the performance of Jetty ? (We have already
> followed
> http://www.eclipse.org/jetty/documentation/9.2.21.v20170120/
> optimizing.html
> recommendation)
>
>     Many thanks,
>
>
> Gérald Reinhart
>
>
> Kelkoo SAS
> Société par Actions Simplifiée
> Au capital de € 4.168.964,30
> Siège social : 158 Ter Rue du Temple 75003 Paris
> 425 093 069 RCS Paris
>
> Ce message et les pièces jointes sont confidentiels et établis à
> l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
> destinataire de ce message, merci de le détruire et d'en avertir
> l'expéditeur.
>



--
Bill Bell
[hidden email]
cell 720-256-8076
Reply | Threaded
Open this post in threaded view
|

Re: [Benchmark SOLR] JETTY VS TOMCAT

Yago Riveiro
Solr run tests with jetty.

I ran in nasty bugs in solr in the past with tomcat.

My advise it’s that speed is only one more metric, robustness and reliability matter too.

--

/Yago Riveiro

On 27 Jan 2017 15:38 +0000, William Bell <[hidden email]>, wrote:

> Did you try:
>
> Set your acceptor count, SelectChannelConnector.setAcceptors(int)
> <http://download.eclipse.org/jetty/stable-7/apidocs/org/eclipse/jetty/server/AbstractConnector.html#setAcceptors%28int%29
> to
> be a a value between 1 and (number_of_cpu_cores - 1).
>
> On Fri, Jan 27, 2017 at 3:22 AM, Gerald Reinhart <[hidden email]
> > wrote:
>
> > Hello,
> >
> > We are migrating our platform
> > from
> > - Solr 5.4.1 hosted by a Tomcat
> > to
> > - Solr 5.4.1 standalone (hosted by Jetty)
> >
> > => Jetty is 15% slower than Tomcat in the same conditions.
> >
> >
> > Here are details about the benchmarks :
> >
> > Context :
> > - Index with 9 000 000 documents
> > - Gatling launch queries extracted from the real traffic
> > - Server : R410 with 16 virtual CPU and 96G mem
> >
> > Results with 20 clients in // during 10 minutes:
> > For Tomcat :
> > - 165 Queries per seconds
> > - 120ms mean response time
> >
> > For Jetty :
> > - 139 Queries per seconds
> > - 142ms mean response time
> >
> > We have checked :
> > - the load of the server => same
> > - the io wait => same
> > - the memory used in the JVM => same
> > - JVM GC settings => same
> >
> > For us, it's a blocker for the migration.
> >
> > Is it a known issue ? (I found that :
> > http://www.asjava.com/jetty/jetty-vs-tomcat-performance-comparison/)
> >
> > How can we improve the performance of Jetty ? (We have already
> > followed
> > http://www.eclipse.org/jetty/documentation/9.2.21.v20170120/
> > optimizing.html
> > recommendation)
> >
> > Many thanks,
> >
> >
> > Gérald Reinhart
> >
> >
> > Kelkoo SAS
> > Société par Actions Simplifiée
> > Au capital de € 4.168.964,30
> > Siège social : 158 Ter Rue du Temple 75003 Paris
> > 425 093 069 RCS Paris
> >
> > Ce message et les pièces jointes sont confidentiels et établis à
> > l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
> > destinataire de ce message, merci de le détruire et d'en avertir
> > l'expéditeur.
> >
>
>
>
> --
> Bill Bell
> [hidden email]
> cell 720-256-8076
Best regards /Yago
Reply | Threaded
Open this post in threaded view
|

[Benchmark SOLR] JETTY VS TOMCAT - Jetty 15% slower - need advice to improve Jetty performance

Gerald Reinhart
In reply to this post by Gerald Reinhart

Hello,

     In addition to the following settings, we have tried to :
         - force Jetty to use more threads
         - put the same GC options as our Tomcat
         - change nb of acceptors and selectors

    and every time Jetty is slower than Tomcat.

    Any advice is welcome

Thanks,

Gérald Reinhart




On 01/27/2017 11:22 AM, Gerald Reinhart wrote:

> Hello,
>
>      We are migrating our platform
>      from
>           - Solr 5.4.1 hosted by a Tomcat
>      to
>           - Solr 5.4.1 standalone (hosted by Jetty)
>
> => Jetty is 15% slower than Tomcat in the same conditions.
>
>
>     Here are details about the benchmarks :
>
>     Context :
>          - Index with 9 000 000 documents
>          - Gatling launch queries extracted from the real traffic
>          - Server :  R410 with 16 virtual CPU and 96G mem
>
>     Results with 20 clients in // during 10 minutes:
>      For Tomcat :
>          - 165 Queries per seconds
>          - 120ms mean response time
>
>      For Jetty :
>          - 139 Queries per seconds
>          - 142ms mean response time
>
> We have checked :
>         - the load of the server => same
>         - the io wait => same
>         - the memory used in the JVM => same
>         - JVM GC settings => same
>
>        For us, it's a blocker for the migration.
>
>        Is it a known issue ? (I found that :
> http://www.asjava.com/jetty/jetty-vs-tomcat-performance-comparison/)
>
>        How can we improve the performance of Jetty ? (We have already
> followed
> http://www.eclipse.org/jetty/documentation/9.2.21.v20170120/optimizing.html
> recommendation)
>
>       Many thanks,
>
>
> Gérald Reinhart
>
>
> Kelkoo SAS
> Société par Actions Simplifiée
> Au capital de € 4.168.964,30
> Siège social : 158 Ter Rue du Temple 75003 Paris
> 425 093 069 RCS Paris
>
> Ce message et les pièces jointes sont confidentiels et établis à l'attention exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce message, merci de le détruire et d'en avertir l'expéditeur.


--

Gérald Reinhart
Software Engineer

<http://www.kelkoo.com/>

*E*[hidden email]
*A*Parc Sud Galaxie, 6, rue des Méridiens, 38130 Echirolles, FR


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce message, merci de le détruire et d'en avertir l'expéditeur.
Reply | Threaded
Open this post in threaded view
|

Re: [Benchmark SOLR] JETTY VS TOMCAT - Jetty 15% slower - need advice to improve Jetty performance

Rick Leir-2
There is a profiling tool in Eclipse that can show you a tree of method
calls, with timing information. I have found this useful in the past to
investigate a performance problem. But it might not help if the problem
only occurs at 165 queries per second (is that true?).

cheers -- Rick


On 2017-01-30 04:02 AM, Gerald Reinhart wrote:

>
> Hello,
>
>     In addition to the following settings, we have tried to :
>         - force Jetty to use more threads
>         - put the same GC options as our Tomcat
>         - change nb of acceptors and selectors
>
>    and every time Jetty is slower than Tomcat.
>
>    Any advice is welcome
>
> Thanks,
>
> Gérald Reinhart
>
>
>
>
> On 01/27/2017 11:22 AM, Gerald Reinhart wrote:
>> Hello,
>>
>>      We are migrating our platform
>>      from
>>           - Solr 5.4.1 hosted by a Tomcat
>>      to
>>           - Solr 5.4.1 standalone (hosted by Jetty)
>>
>> => Jetty is 15% slower than Tomcat in the same conditions.
>>
>>
>>     Here are details about the benchmarks :
>>
>>     Context :
>>          - Index with 9 000 000 documents
>>          - Gatling launch queries extracted from the real traffic
>>          - Server :  R410 with 16 virtual CPU and 96G mem
>>
>>     Results with 20 clients in // during 10 minutes:
>>      For Tomcat :
>>          - 165 Queries per seconds
>>          - 120ms mean response time
>>
>>      For Jetty :
>>          - 139 Queries per seconds
>>          - 142ms mean response time
>>
>> We have checked :
>>         - the load of the server => same
>>         - the io wait => same
>>         - the memory used in the JVM => same
>>         - JVM GC settings => same
>>
>>        For us, it's a blocker for the migration.
>>
>>        Is it a known issue ? (I found that :
>> http://www.asjava.com/jetty/jetty-vs-tomcat-performance-comparison/)
>>
>>        How can we improve the performance of Jetty ? (We have already
>> followed
>> http://www.eclipse.org/jetty/documentation/9.2.21.v20170120/optimizing.html 
>>
>> recommendation)
>>
>>       Many thanks,
>>
>>
>> Gérald Reinhart
>>
>>
>> Kelkoo SAS
>> Société par Actions Simplifiée
>> Au capital de € 4.168.964,30
>> Siège social : 158 Ter Rue du Temple 75003 Paris
>> 425 093 069 RCS Paris
>>
>> Ce message et les pièces jointes sont confidentiels et établis à
>> l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
>> destinataire de ce message, merci de le détruire et d'en avertir
>> l'expéditeur.
>
>
> --
>
> Gérald Reinhart
> Software Engineer
>
> <http://www.kelkoo.com/>
>
> *E*[hidden email]
> *A*Parc Sud Galaxie, 6, rue des Méridiens, 38130 Echirolles, FR
>
>
> Kelkoo SAS
> Société par Actions Simplifiée
> Au capital de € 4.168.964,30
> Siège social : 158 Ter Rue du Temple 75003 Paris
> 425 093 069 RCS Paris
>
> Ce message et les pièces jointes sont confidentiels et établis à
> l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
> destinataire de ce message, merci de le détruire et d'en avertir
> l'expéditeur.

Reply | Threaded
Open this post in threaded view
|

Re: [Benchmark SOLR] JETTY VS TOMCAT - Jetty 15% slower - need advice to improve Jetty performance

Gerald Reinhart


We have done some profiling with Visualvm, but nothing obvious appeared.

Thank Rick for the advice.

Gérald Reinhart


On 02/01/2017 11:17 AM, Rick Leir wrote:

> There is a profiling tool in Eclipse that can show you a tree of method
> calls, with timing information. I have found this useful in the past to
> investigate a performance problem. But it might not help if the problem
> only occurs at 165 queries per second (is that true?).
>
> cheers -- Rick
>
>
> On 2017-01-30 04:02 AM, Gerald Reinhart wrote:
>> Hello,
>>
>>      In addition to the following settings, we have tried to :
>>          - force Jetty to use more threads
>>          - put the same GC options as our Tomcat
>>          - change nb of acceptors and selectors
>>
>>     and every time Jetty is slower than Tomcat.
>>
>>     Any advice is welcome
>>
>> Thanks,
>>
>> Gérald Reinhart
>>
>>
>>
>>
>> On 01/27/2017 11:22 AM, Gerald Reinhart wrote:
>>> Hello,
>>>
>>>       We are migrating our platform
>>>       from
>>>            - Solr 5.4.1 hosted by a Tomcat
>>>       to
>>>            - Solr 5.4.1 standalone (hosted by Jetty)
>>>
>>> => Jetty is 15% slower than Tomcat in the same conditions.
>>>
>>>
>>>      Here are details about the benchmarks :
>>>
>>>      Context :
>>>           - Index with 9 000 000 documents
>>>           - Gatling launch queries extracted from the real traffic
>>>           - Server :  R410 with 16 virtual CPU and 96G mem
>>>
>>>      Results with 20 clients in // during 10 minutes:
>>>       For Tomcat :
>>>           - 165 Queries per seconds
>>>           - 120ms mean response time
>>>
>>>       For Jetty :
>>>           - 139 Queries per seconds
>>>           - 142ms mean response time
>>>
>>> We have checked :
>>>          - the load of the server => same
>>>          - the io wait => same
>>>          - the memory used in the JVM => same
>>>          - JVM GC settings => same
>>>
>>>         For us, it's a blocker for the migration.
>>>
>>>         Is it a known issue ? (I found that :
>>> http://www.asjava.com/jetty/jetty-vs-tomcat-performance-comparison/)
>>>
>>>         How can we improve the performance of Jetty ? (We have already
>>> followed
>>> http://www.eclipse.org/jetty/documentation/9.2.21.v20170120/optimizing.html
>>>
>>> recommendation)
>>>
>>>        Many thanks,
>>>
>>>
>>> Gérald Reinhart
>>>
>>>
>>> Kelkoo SAS
>>> Société par Actions Simplifiée
>>> Au capital de € 4.168.964,30
>>> Siège social : 158 Ter Rue du Temple 75003 Paris
>>> 425 093 069 RCS Paris
>>>
>>> Ce message et les pièces jointes sont confidentiels et établis à
>>> l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
>>> destinataire de ce message, merci de le détruire et d'en avertir
>>> l'expéditeur.
>>
>> --
>>
>> Gérald Reinhart
>> Software Engineer
>>
>> <http://www.kelkoo.com/>
>>
>> *E*[hidden email]
>> *A*Parc Sud Galaxie, 6, rue des Méridiens, 38130 Echirolles, FR
>>
>>
>> Kelkoo SAS
>> Société par Actions Simplifiée
>> Au capital de € 4.168.964,30
>> Siège social : 158 Ter Rue du Temple 75003 Paris
>> 425 093 069 RCS Paris
>>
>> Ce message et les pièces jointes sont confidentiels et établis à
>> l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
>> destinataire de ce message, merci de le détruire et d'en avertir
>> l'expéditeur.


--

Gérald Reinhart
Software Engineer

<http://www.kelkoo.com/>

*E*[hidden email]
*A*Parc Sud Galaxie, 6, rue des Méridiens, 38130 Echirolles, FR


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce message, merci de le détruire et d'en avertir l'expéditeur.