[jira] Created: (LUCENE-2706) ByteBlockPool should allow different block sizes

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[jira] Created: (LUCENE-2706) ByteBlockPool should allow different block sizes

Hudson (Jira)
ByteBlockPool should allow different block sizes

                 Key: LUCENE-2706
                 URL: https://issues.apache.org/jira/browse/LUCENE-2706
             Project: Lucene - Java
          Issue Type: Improvement
    Affects Versions: 4.0
            Reporter: Simon Willnauer
            Assignee: Simon Willnauer

Currently ByteBlockPool has a fixed byteblock size of 1 << 15 bytes. This limit also makes BytesRefHash bound to 32kb buffers which is not ideal in many cases. For instance in MTQ (LUCENE-2960) we might want to have a smaller block size of 1024 or 2048 for more efficiency. While this example  suffers from another limitation since the blockSize used in TermsHashPerField is 32kb which in turn defines the size limit for a single term in the index. To make effective use of different byte block sizes we need to find a solution for this problem too but I think index statistics are on the way and max term length for a field could also make merges more efficient if we can alter the byte block size especially for fields with very short terms.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]