Compound file don't work with lucene 3.3 or 3.4

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Compound file don't work with lucene 3.3 or 3.4

liugang
I use lucene 3.4 in my search app.  in default config, after indexing, my index dir has several file such as *.tii, *.tis ..., and cfs file doesn't exist.

Then I use setUseCompoundFile(true) of logByteSizeMergePolicy, and result is same.

I remember it's good with lucene 2.x.  has anybody same problem? Appreciate for any response.

发自我的 iPad
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Compound file don't work with lucene 3.3 or 3.4

Uwe Schindler
Hi,

Lucene only uses CFS for very small segments, that don't exceed a specific
percentage of the index size. See the method LogMergePolicy.setNoCFSRatio
[http://lucene.apache.org/java/3_4_0/api/all/org/apache/lucene/index/LogMerg
ePolicy.html#setNoCFSRatio(double)].

The default is 10%, every segment larger than 0.1 of total index size will
not be made compound:
[http://lucene.apache.org/java/3_4_0/api/all/org/apache/lucene/index/LogMerg
ePolicy.html#DEFAULT_NO_CFS_RATIO]

In general for merge performance reason you should not change those values,
unless you are running out of file descriptors! If you have a static index
that never changes (e.g. that will be hosted on a CD-ROM), you can
forceMerge(1) it to one single segment with CFSRatio=1.0, in all other cases
it's not a good idea, especially if you do realtime indexing; in that case
you should use the new TieredMergePolicy not the old LogMergePolicy.

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [hidden email]


> -----Original Message-----
> From: liugang [mailto:[hidden email]]
> Sent: Sunday, November 20, 2011 5:30 AM
> To: [hidden email]
> Subject: Compound file don't work with lucene 3.3 or 3.4
>
> I use lucene 3.4 in my search app.  in default config, after indexing, my
index
> dir has several file such as *.tii, *.tis ..., and cfs file doesn't exist.
>
> Then I use setUseCompoundFile(true) of logByteSizeMergePolicy, and result
is
> same.
>
> I remember it's good with lucene 2.x.  has anybody same problem?
Appreciate
> for any response.
>
> 发自我的 iPad
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re:RE: Compound file don't work with lucene 3.3 or 3.4

liugang
thanks for uwe's answer.


--
gang liu
email: [hidden email]



At 2011-11-20 17:44:06,"Uwe Schindler" <[hidden email]> wrote:

>Hi,
>
>Lucene only uses CFS for very small segments, that don't exceed a specific
>percentage of the index size. See the method LogMergePolicy.setNoCFSRatio
>[http://lucene.apache.org/java/3_4_0/api/all/org/apache/lucene/index/LogMerg
>ePolicy.html#setNoCFSRatio(double)].
>
>The default is 10%, every segment larger than 0.1 of total index size will
>not be made compound:
>[http://lucene.apache.org/java/3_4_0/api/all/org/apache/lucene/index/LogMerg
>ePolicy.html#DEFAULT_NO_CFS_RATIO]
>
>In general for merge performance reason you should not change those values,
>unless you are running out of file descriptors! If you have a static index
>that never changes (e.g. that will be hosted on a CD-ROM), you can
>forceMerge(1) it to one single segment with CFSRatio=1.0, in all other cases
>it's not a good idea, especially if you do realtime indexing; in that case
>you should use the new TieredMergePolicy not the old LogMergePolicy.
>
>Uwe
>
>-----
>Uwe Schindler
>H.-H.-Meier-Allee 63, D-28213 Bremen
>http://www.thetaphi.de
>eMail: [hidden email]
>
>
>> -----Original Message-----
>> From: liugang [mailto:[hidden email]]
>> Sent: Sunday, November 20, 2011 5:30 AM
>> To: [hidden email]
>> Subject: Compound file don't work with lucene 3.3 or 3.4
>>
>> I use lucene 3.4 in my search app.  in default config, after indexing, my
>index
>> dir has several file such as *.tii, *.tis ..., and cfs file doesn't exist.
>>
>> Then I use setUseCompoundFile(true) of logByteSizeMergePolicy, and result
>is
>> same.
>>
>> I remember it's good with lucene 2.x.  has anybody same problem?
>Appreciate
>> for any response.
>>
>> 发自我的 iPad
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: [hidden email]
>For additional commands, e-mail: [hidden email]
>