when was the document number initially written into .frq file?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

when was the document number initially written into .frq file?

Charlie-24
Hello,

Would any developer please give me a hint of when the document number was
initially written into .frq file?

From: //it is not really write the doc# in writePostings()

final class DocumentWriter
  private final void writePostings(Posting[] postings, String segment)
        int postingFreq = posting.freq;
        if (postingFreq == 1)                             // optimize freq=1
          freq.writeVInt(1);                      // set low bit of doc num.
        else {
          freq.writeVInt(0);                      // the document number
          freq.writeVInt(postingFreq);                    // frequency in doc
        }

//it is write the doc# in appendPostings()

final class SegmentMerger
  private final int appendPostings(SegmentMergeInfo[] smis, int n)

        int docCode = (doc - lastDoc) << 1;       // use low bit to flag freq=1
        lastDoc = doc;

        int freq = postings.freq();
        if (freq == 1) {
          freqOutput.writeVInt(docCode | 1);      // write doc & freq=1
        } else {
          freqOutput.writeVInt(docCode);          // write doc
          freqOutput.writeVInt(freq);             // write frequency in doc
        }

//but then I am further confused that in order to call
int doc = postings.doc(); in appendPostings(),
the doc# should already been written.

Chicken-egg-chicken-egg ...

Should there be another place for the initial writing of the doc# ?

--
Thanks for your advice,
 Charlie


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: when was the document number initially written into .frq file?

jian chen
It is in DocumentWriter.java class.

Look at writePostings(...) method.

Here are the lines:

    // add an entry to the freq file
    int f = posting.freq;
    if (f == 1)                  // optimize freq=1
      freq.writeVInt(1);              // set low bit of doc num.
    else {
      freq.writeVInt(0);              // the document number
      freq.writeVInt(f);              // frequency in doc
    }

Any other question?

Jian

On 5/6/06, Charlie <[hidden email]> wrote:

>
> Hello,
>
> Would any developer please give me a hint of when the document number was
> initially written into .frq file?
>
> From: //it is not really write the doc# in writePostings()
>
> final class DocumentWriter
>   private final void writePostings(Posting[] postings, String segment)
>         int postingFreq = posting.freq;
>         if (postingFreq == 1)                             // optimize
> freq=1
>           freq.writeVInt(1);                      // set low bit of doc
> num.
>         else {
>           freq.writeVInt(0);                      // the document number
>           freq.writeVInt(postingFreq);                    // frequency in
> doc
>         }
>
> //it is write the doc# in appendPostings()
>
> final class SegmentMerger
>   private final int appendPostings(SegmentMergeInfo[] smis, int n)
>
>         int docCode = (doc - lastDoc) << 1;       // use low bit to flag
> freq=1
>         lastDoc = doc;
>
>         int freq = postings.freq();
>         if (freq == 1) {
>           freqOutput.writeVInt(docCode | 1);      // write doc & freq=1
>         } else {
>           freqOutput.writeVInt(docCode);          // write doc
>           freqOutput.writeVInt(freq);             // write frequency in
> doc
>         }
>
> //but then I am further confused that in order to call
> int doc = postings.doc(); in appendPostings(),
> the doc# should already been written.
>
> Chicken-egg-chicken-egg ...
>
> Should there be another place for the initial writing of the doc# ?
>
> --
> Thanks for your advice,
> Charlie
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: when was the document number initially written into .frq file?

jian chen
Looking at your email again.

You are confusing the initial writing of postings with the segment merging.

Once the doc number is written, the .frq file is not changed. The segment
merge process will write to a new .frq file.

Make sense?

Jian

On 5/8/06, jian chen <[hidden email]> wrote:

>
> It is in DocumentWriter.java class.
>
> Look at writePostings(...) method.
>
> Here are the lines:
>
>     // add an entry to the freq file
>     int f = posting.freq;
>     if (f == 1)                  // optimize freq=1
>
>       freq.writeVInt(1);              // set low bit of doc num.
>     else {
>       freq.writeVInt(0);              // the document number
>       freq.writeVInt(f);              // frequency in doc
>     }
>
> Any other question?
>
> Jian
>
>
> On 5/6/06, Charlie <[hidden email]> wrote:
> >
> > Hello,
> >
> > Would any developer please give me a hint of when the document number
> > was
> > initially written into .frq file?
> >
> > From: //it is not really write the doc# in writePostings()
> >
> > final class DocumentWriter
> >   private final void writePostings(Posting[] postings, String segment)
> >         int postingFreq = posting.freq;
> >         if (postingFreq == 1)                             // optimize
> > freq=1
> >           freq.writeVInt(1);                      // set low bit of doc
> > num.
> >         else {
> >           freq.writeVInt(0);                      // the document number
> >           freq.writeVInt(postingFreq);                    // frequency
> > in doc
> >         }
> >
> > //it is write the doc# in appendPostings()
> >
> > final class SegmentMerger
> >   private final int appendPostings(SegmentMergeInfo[] smis, int n)
> >
> >         int docCode = (doc - lastDoc) << 1;       // use low bit to flag
> > freq=1
> >         lastDoc = doc;
> >
> >         int freq = postings.freq();
> >         if (freq == 1) {
> >           freqOutput.writeVInt(docCode | 1);      // write doc & freq=1
> >         } else {
> >           freqOutput.writeVInt(docCode);          // write doc
> >           freqOutput.writeVInt(freq);             // write frequency in
> > doc
> >         }
> >
> > //but then I am further confused that in order to call
> > int doc = postings.doc(); in appendPostings(),
> > the doc# should already been written.
> >
> > Chicken-egg-chicken-egg ...
> >
> > Should there be another place for the initial writing of the doc# ?
> >
> > --
> > Thanks for your advice,
> > Charlie
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
> >
>