Number of file descriptors for "typical" index life cycle.

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Number of file descriptors for "typical" index life cycle.

Lukáš Vlček
Hi,

Lucene consumes a lot of file descriptors when managing index. I would like
to understand the basic math in estimating fd limits. Can you please let me
know if the below is basically valid assumption?

- an "average" Lucene index can break down to 40-50 segments [1]
- each segment can consists of around 10 to 15 files [2]

This can be up to 750 individual files per Lucene index (non-optimized).
That would mean 750 fd.

Now, does segment merges add any counts to needed fd as well? I mean, if
let's say 10 segments are being merged, how many fd this merge process
needs? Does a single merge consume more fd then what is needed for the
resulting segment?

Thanks,
Lukas

[1]
http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html
[2]
http://lucene.apache.org/core/4_3_0/core/org/apache/lucene/codecs/lucene42/package-summary.html#Overview