segment ? new segment after a commit

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

segment ? new segment after a commit

sunnyShiny06
Hi,

I just made an update :
Indexing completed. Added/Updated: 6327 documents. Deleted 0 documents.

But I don"t get why it doesn't just add a new segment instead of change all the segment.

ls data/index/
_1zf.fdt  _1zf.fdx  _1zf.fnm  _1zf.frq  _1zf.nrm  _1zf.prx  _1zf.tii  _1zf.tis  _1zf.tvd  _1zf.tvf  _1zf.tvx  segments.gen  segments_1o

during the update
ls data/index/
_1zf.fdt  _1zf.fnm  _1zf.nrm  _1zf.tii  _1zf.tvd  _1zf.tvx    _1zg.fdt  _1zg.fnm  _1zg.nrm  _1zg.tii  _1zg.tvd  _1zg.tvx  _1zh.frq  _1zh.prx  _1zh.tis  _1zi.fdx  _1zi.frq  _1zi.tii  segments.gen
_1zf.fdx  _1zf.frq  _1zf.prx  _1zf.tis  _1zf.tvf  _1zf_1.del  _1zg.fdx  _1zg.frq  _1zg.prx  _1zg.tis  _1zg.tvf  _1zh.fnm  _1zh.nrm  _1zh.tii  _1zi.fdt  _1zi.fnm  _1zi.prx  _1zi.tis  segments_1o

ls data/index/
_1zi.fdt  _1zi.fdx  _1zi.fnm  _1zi.frq  _1zi.nrm  _1zi.prx  _1zi.tii  _1zi.tis  _1zi.tvd  _1zi.tvf  _1zi.tvx  segments.gen  segments_1p


This is my conf:

   <mergeFactor>15</mergeFactor>
   
    <maxBufferedDocs>30000</maxBufferedDocs>
   
    <maxMergeDocs>2147483647</maxMergeDocs>
    <maxFieldLength>10000</maxFieldLength>
    <writeLockTimeout>1000</writeLockTimeout>
    <commitLockTimeout>1000000</commitLockTimeout>

   
   
   
   

   
   

   
    <lockType>single</lockType>
  </indexDefaults>

  <mainIndex>
   
    <useCompoundFile>false</useCompoundFile>
    <mergeFactor>15</mergeFactor>
   
    <maxBufferedDocs>50000</maxBufferedDocs>
    <maxMergeDocs>2147483647</maxMergeDocs>
    <maxFieldLength>10000</maxFieldLength>

Reply | Threaded
Open this post in threaded view
|

Re: segment ? new segment after a commit

Ted Dunning
Here is a link to a talk that Doug gave describing the basics of the
indexing process.  It should answer your questions.

http://lucene.sourceforge.net/talks/pisa/

The basic answer is that adding documents increases the size of the smaller
files until they get big enough to merge and then when the merged result
gets bigger, it gets merged to the next file.

If you add a small number of new documents, measured as a fraction of all
documents, then only a few files will change.  If you had many documents,
then many files will change.

On Thu, Apr 16, 2009 at 6:51 AM, sunnyfr <[hidden email]> wrote:

>
>
> I just made an update :
> Indexing completed. Added/Updated: 6327 documents. Deleted 0 documents.
>
> But I don"t get why it doesn't just add a new segment instead of change all
> the segment.
>
>