[jira] [Comment Edited] (SOLR-12885) BinaryResponseWriter (javabin format) should directly copy from Bytesref to output

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Comment Edited] (SOLR-12885) BinaryResponseWriter (javabin format) should directly copy from Bytesref to output

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/SOLR-12885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709641#comment-16709641 ]

Noble Paul edited comment on SOLR-12885 at 12/6/18 12:01 AM:
-------------------------------------------------------------

some numbers

Test docs
 10 docs with 6 String field

Test 1: cache is disabled .
 * Every query reads from stored fields
 * Each field creates an appropriate object of string/Utf8CharSequence depending on what is set

Test 2: cache is enabled .
 * Every query reads from document cache
 * Only serialization performance is tested

{code:java}
Test:1
Total queries:  10K
NO docs from document cache
------------------------------------------------------------------------
Using UTF8 : false
time taken : 10890
Total Strings created from stored fields : 600000
Total UTF8 created from storedfields : 0
Total java Strings serialized : 780000
Total UTF8 serialized  : 0

----------------------------------------------------------------------------

Using UTF8 : true
Using DOC cache : false
time taken : 6550
Total Strings created from stored fields : 0
Total UTF8 created from storedfields : 600000
Total java Strings serialized : 180000
Total UTF8 serialized  : 600000

===============================================
Test:2
Total queries:  10K
ALL docs from document cache
===============================================
Using UTF8 : false
Using DOC cache : true
time taken : 10335
Total Strings created from stored fields : 0
Total UTF8 created from storedfields : 0
Total java Strings serialized : 180000
Total UTF8 serialized  : 600000

------------------------------------------------------------------------
Using UTF8 : true
Using DOC cache : true
time taken : 5551
Total Strings created from stored fields : 0
Total UTF8 created from storedfields : 0
Total java Strings serialized : 180000
Total UTF8 serialized  : 600000
------------------------------------------------------------------------
{code}


was (Author: noble.paul):
some numbers

Test docs
10 docs with 6 String field

Test 1: cache is disabled .
* Every query reads from stored fields
* Each field creates an appropriate object of string/Utf8CharSequence depending on what is set

Test 2: cache is enabled .
* Every query reads from document cache
* Only serialization performance is tested

{code}
Test:1
Total queries:  10K
NO docs from document cache
------------------------------------------------------------------------
use : String
time taken : 12932
Total Strings created : 600000
Total UTF8 created : 0
JavaBinCodec.utf16_to_utf8 :  780000
FastOutputStream.writeUtf8CharSeqs : 0

--------------------------------------------------------------------------------
use : Utf8CharSequence
time taken : 7801
Total Strings created: 0
Total UTF8 created: 600000
JavaBinCodec.utf16_to_utf8 :  180000
FastOutputStream.writeUtf8CharSeqs : 600000

===============================================
Test:2
Total queries:  10K
ALL docs from document cache
===============================================
use : String
time taken : 10362
Total Strings created : 600000
Total UTF8 created : 0
JavaBinCodec.utf16_to_utf8 : 780000
FastOutputStream.writeUtf8CharSeqs : 0
------------------------------------------------------------------------
use: Utf8CharSquence
time taken : 6104
Total Strings created : 0
Total UTF8 created : 600000
JavaBinCodec.utf16_to_utf8 : 180000
FastOutputStream.writeUtf8CharSeqs : 600000
{code}



> BinaryResponseWriter (javabin format) should directly copy from Bytesref to output
> ----------------------------------------------------------------------------------
>
>                 Key: SOLR-12885
>                 URL: https://issues.apache.org/jira/browse/SOLR-12885
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public)
>            Reporter: Noble Paul
>            Assignee: Noble Paul
>            Priority: Major
>         Attachments: SOLR-12885.patch, SOLR-12885.patch, SOLR-12885.patch, SOLR-12885.patch
>
>
> The format format in which bytes are stored in {{BytesRef}} and the javabin string format are both the same. We don't need to convert the string/text fields from {{BytesRef}} to String and back to UTF8 
> {{Now a String/Text field is read and written out as follows}}
> {{luceneindex(UTF8 bytes) --> UTF16 (char[]) --> new String() a copy of UTF16 char[] -->  UTF8bytes(javabin format)}}
> This does not add a new type to javabin. It's encoded as String in the serialized data. When it is deserialized, you get a String back



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]