XML vs JSON writer performance issues

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

XML vs JSON writer performance issues

Jérôme Etévé-2
Hi all.
  I'm currently running some tests with solr on a small index and I
noticed a big difference on the response time of queries depending on
the use of XML or json as a response format.
In average, my test queries (including http connections open and close
) takes 6 ms to perform when I ask for XML and they take 30 ms when I
ask for JSON.
When I'm running lots of test clients at the same time, the same
factor 30/6 seems to apply.

I looked at the code and didn't see any major difference between the
two writers.
I'd rather use json instead of XML, but that performance issue prevents me to.

I'm using  apache-solr-1.2.0 / apache-tomcat-6.0.13 / java version
"1.5.0_09" (Sun)

Thanks for any comments or help.

--
Jerome Eteve.
[hidden email]
http://www.eteve.net
Reply | Threaded
Open this post in threaded view
|

Re: XML vs JSON writer performance issues

Yonik Seeley-2
On 6/26/07, Jérôme Etévé <[hidden email]> wrote:
>   I'm currently running some tests with solr on a small index and I
> noticed a big difference on the response time of queries depending on
> the use of XML or json as a response format.
> In average, my test queries (including http connections open and close
> ) takes 6 ms to perform when I ask for XML and they take 30 ms when I
> ask for JSON.

Wow, that's a surprise.
The only thing I can figure is that perhaps during the string escaping
the JSON writer is writing to the stream character-by-character.
Could you try the python writer and see if there is a speed
difference?  It uses a StringBuilder when escaping the string.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: XML vs JSON writer performance issues

Jérôme Etévé-2
On 6/26/07, Yonik Seeley <[hidden email]> wrote:

> On 6/26/07, Jérôme Etévé <[hidden email]> wrote:
> >   I'm currently running some tests with solr on a small index and I
> > noticed a big difference on the response time of queries depending on
> > the use of XML or json as a response format.
> > In average, my test queries (including http connections open and close
> > ) takes 6 ms to perform when I ask for XML and they take 30 ms when I
> > ask for JSON.
>
> Wow, that's a surprise.
> The only thing I can figure is that perhaps during the string escaping
> the JSON writer is writing to the stream character-by-character.
> Could you try the python writer and see if there is a speed
> difference?  It uses a StringBuilder when escaping the string.

I just tried the python writer and it's as fast as XML is.  I'm still
looking at the code trying to point out the reason of that.
Thanks for any help.

J

--
Jerome Eteve.
[hidden email]
http://www.eteve.net
Reply | Threaded
Open this post in threaded view
|

Re: XML vs JSON writer performance issues

Yonik Seeley-2
On 6/26/07, Jérôme Etévé <[hidden email]> wrote:
> I just tried the python writer and it's as fast as XML is.  I'm still
> looking at the code trying to point out the reason of that.
> Thanks for any help.

It's Java's braindead Reader/Writer implementations.  If you want
really good performance, one is forced to do their own buffering, even
for a BufferedWriter (I assume the PrintWriter obtained from
ServletResponse.getWriter() is buffered).

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: XML vs JSON writer performance issues

Yonik Seeley-2
It would be helpful if you could try out the patch at
https://issues.apache.org/jira/browse/SOLR-276

-Yonik


On 6/26/07, Yonik Seeley <[hidden email]> wrote:

> On 6/26/07, Jérôme Etévé <[hidden email]> wrote:
> > I just tried the python writer and it's as fast as XML is.  I'm still
> > looking at the code trying to point out the reason of that.
> > Thanks for any help.
>
> It's Java's braindead Reader/Writer implementations.  If you want
> really good performance, one is forced to do their own buffering, even
> for a BufferedWriter (I assume the PrintWriter obtained from
> ServletResponse.getWriter() is buffered).
>
> -Yonik
>
Reply | Threaded
Open this post in threaded view
|

Re: XML vs JSON writer performance issues

James liu-2
XML data should bigger thant JSON data, and transfer quicker than JSON..

it surprised me.

2007/6/27, Yonik Seeley <[hidden email]>:

>
> It would be helpful if you could try out the patch at
> https://issues.apache.org/jira/browse/SOLR-276
>
> -Yonik
>
>
> On 6/26/07, Yonik Seeley <[hidden email]> wrote:
> > On 6/26/07, Jérôme Etévé <[hidden email]> wrote:
> > > I just tried the python writer and it's as fast as XML is.  I'm still
> > > looking at the code trying to point out the reason of that.
> > > Thanks for any help.
> >
> > It's Java's braindead Reader/Writer implementations.  If you want
> > really good performance, one is forced to do their own buffering, even
> > for a BufferedWriter (I assume the PrintWriter obtained from
> > ServletResponse.getWriter() is buffered).
> >
> > -Yonik
> >
>



--
regards
jl
Reply | Threaded
Open this post in threaded view
|

Re: XML vs JSON writer performance issues

Jérôme Etévé-2
>
> 2007/6/27, Yonik Seeley <[hidden email]>:
> >
> > It would be helpful if you could try out the patch at
> > https://issues.apache.org/jira/browse/SOLR-276
> >
> > -Yonik

I just tryed it out and it works. json output is now as fast as xml !
Well done :) thank you !

J.

--
Jerome Eteve.
[hidden email]
http://jerome.eteve.free.fr/