[jira] Created: (LUCENE-706) Index File Format - Example for frequency file .frq is wrong

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (LUCENE-706) Index File Format - Example for frequency file .frq is wrong

JIRA jira@apache.org
Index File Format - Example for frequency file .frq is wrong
------------------------------------------------------------

                 Key: LUCENE-706
                 URL: http://issues.apache.org/jira/browse/LUCENE-706
             Project: Lucene - Java
          Issue Type: Improvement
          Components: Website
         Environment: not applicable
            Reporter: Doron Cohen
         Assigned To: Doron Cohen
            Priority: Trivial


Reported by Johan Stuyts - http://www.nabble.com/Possible-documentation-error--p7012445.html -

Frequency file example says:

     For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
         15, 22, 3

It should be:

     For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
         15, 8, 3




--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-706) Index File Format - Example for frequency file .frq is wrong

JIRA jira@apache.org
     [ http://issues.apache.org/jira/browse/LUCENE-706?page=all ]

Doron Cohen updated LUCENE-706:
-------------------------------

    Attachment: file-format-frq-example.patch

example fixed

> Index File Format - Example for frequency file .frq is wrong
> ------------------------------------------------------------
>
>                 Key: LUCENE-706
>                 URL: http://issues.apache.org/jira/browse/LUCENE-706
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Website
>         Environment: not applicable
>            Reporter: Doron Cohen
>         Assigned To: Doron Cohen
>            Priority: Trivial
>         Attachments: file-format-frq-example.patch
>
>
> Reported by Johan Stuyts - http://www.nabble.com/Possible-documentation-error--p7012445.html -
> Frequency file example says:
>      For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
>          15, 22, 3
> It should be:
>      For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
>          15, 8, 3

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-706) Index File Format - Example for frequency file .frq is wrong

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
     [ http://issues.apache.org/jira/browse/LUCENE-706?page=all ]

Doron Cohen updated LUCENE-706:
-------------------------------

    Lucene Fields: [New, Patch Available]  (was: [New])

> Index File Format - Example for frequency file .frq is wrong
> ------------------------------------------------------------
>
>                 Key: LUCENE-706
>                 URL: http://issues.apache.org/jira/browse/LUCENE-706
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Website
>         Environment: not applicable
>            Reporter: Doron Cohen
>         Assigned To: Doron Cohen
>            Priority: Trivial
>         Attachments: file-format-frq-example.patch
>
>
> Reported by Johan Stuyts - http://www.nabble.com/Possible-documentation-error--p7012445.html -
> Frequency file example says:
>      For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
>          15, 22, 3
> It should be:
>      For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
>          15, 8, 3

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-706) Index File Format - Example for frequency file .frq is wrong

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
    [ http://issues.apache.org/jira/browse/LUCENE-706?page=comments#action_12446948 ]
           
Grant Ingersoll commented on LUCENE-706:
----------------------------------------

Just to double check the math:
From the Website:
"DocDelta determines both the document number and the frequency. In particular, DocDelta/2 is the difference between this document number and the previous document number (or zero when this is the first document in a TermFreqs). When DocDelta is odd, the frequency is one. When DocDelta is even, the frequency is read as another VInt."

So, 15 is the correct start since 15 /2 as an int is 7 and the frequency is one.  Then the difference between doc 7 and 11 is 4, so the next value should be 8 (since DocDelta/2 = 11 - 7, which is even, meaning the frequency is the next VInt, in this case 3, so I would concur.



> Index File Format - Example for frequency file .frq is wrong
> ------------------------------------------------------------
>
>                 Key: LUCENE-706
>                 URL: http://issues.apache.org/jira/browse/LUCENE-706
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Website
>         Environment: not applicable
>            Reporter: Doron Cohen
>         Assigned To: Doron Cohen
>            Priority: Trivial
>         Attachments: file-format-frq-example.patch
>
>
> Reported by Johan Stuyts - http://www.nabble.com/Possible-documentation-error--p7012445.html -
> Frequency file example says:
>      For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
>          15, 22, 3
> It should be:
>      For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
>          15, 8, 3

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Assigned: (LUCENE-706) Index File Format - Example for frequency file .frq is wrong

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
     [ http://issues.apache.org/jira/browse/LUCENE-706?page=all ]

Grant Ingersoll reassigned LUCENE-706:
--------------------------------------

    Assignee: Grant Ingersoll  (was: Doron Cohen)

> Index File Format - Example for frequency file .frq is wrong
> ------------------------------------------------------------
>
>                 Key: LUCENE-706
>                 URL: http://issues.apache.org/jira/browse/LUCENE-706
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Website
>         Environment: not applicable
>            Reporter: Doron Cohen
>         Assigned To: Grant Ingersoll
>            Priority: Trivial
>         Attachments: file-format-frq-example.patch
>
>
> Reported by Johan Stuyts - http://www.nabble.com/Possible-documentation-error--p7012445.html -
> Frequency file example says:
>      For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
>          15, 22, 3
> It should be:
>      For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
>          15, 8, 3

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-706) Index File Format - Example for frequency file .frq is wrong

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
    [ http://issues.apache.org/jira/browse/LUCENE-706?page=comments#action_12447042 ]
           
Doron Cohen commented on LUCENE-706:
------------------------------------

Right -

   15  =  2 * 7 + 1    --> doc 7 with freq 1
    8   =  2 * (11 - 7) --> doc 11 with frequency > 1
    3                --> frequency = 3 for doc 11

.frq file actual content for similar case also agrees with that, it starts like this (Hex):

     0D 08  03 01 03

(note: Hex:  0D = 15.)




> Index File Format - Example for frequency file .frq is wrong
> ------------------------------------------------------------
>
>                 Key: LUCENE-706
>                 URL: http://issues.apache.org/jira/browse/LUCENE-706
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Website
>         Environment: not applicable
>            Reporter: Doron Cohen
>         Assigned To: Grant Ingersoll
>            Priority: Trivial
>         Attachments: file-format-frq-example.patch
>
>
> Reported by Johan Stuyts - http://www.nabble.com/Possible-documentation-error--p7012445.html -
> Frequency file example says:
>      For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
>          15, 22, 3
> It should be:
>      For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
>          15, 8, 3

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-706) Index File Format - Example for frequency file .frq is wrong

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
    [ http://issues.apache.org/jira/browse/LUCENE-706?page=comments#action_12447044 ]
           
Steven Rowe commented on LUCENE-706:
------------------------------------

Hex: 0D is NOT the same as decimal 15.  0Dh = 13d.  15d = 0Fh.

> Index File Format - Example for frequency file .frq is wrong
> ------------------------------------------------------------
>
>                 Key: LUCENE-706
>                 URL: http://issues.apache.org/jira/browse/LUCENE-706
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Website
>         Environment: not applicable
>            Reporter: Doron Cohen
>         Assigned To: Grant Ingersoll
>            Priority: Trivial
>         Attachments: file-format-frq-example.patch
>
>
> Reported by Johan Stuyts - http://www.nabble.com/Possible-documentation-error--p7012445.html -
> Frequency file example says:
>      For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
>          15, 22, 3
> It should be:
>      For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
>          15, 8, 3

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-706) Index File Format - Example for frequency file .frq is wrong

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
    [ http://issues.apache.org/jira/browse/LUCENE-706?page=comments#action_12447049 ]
           
Doron Cohen commented on LUCENE-706:
------------------------------------

Right, sorry, copied that hex data from an .frq of an index with a different example, where the frequencies were 1 in doc 6 and 3 in doc 10, so there you would get 2 * 6 + 1 = 13.

For the correct example of freq 1 in doc 7 and 3 in doc 11 the .frq content is  0F 08 03  as it should be.

(Meaning that the documentatin should still be fixed...;-)


> Index File Format - Example for frequency file .frq is wrong
> ------------------------------------------------------------
>
>                 Key: LUCENE-706
>                 URL: http://issues.apache.org/jira/browse/LUCENE-706
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Website
>         Environment: not applicable
>            Reporter: Doron Cohen
>         Assigned To: Grant Ingersoll
>            Priority: Trivial
>         Attachments: file-format-frq-example.patch
>
>
> Reported by Johan Stuyts - http://www.nabble.com/Possible-documentation-error--p7012445.html -
> Frequency file example says:
>      For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
>          15, 22, 3
> It should be:
>      For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
>          15, 8, 3

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Resolved: (LUCENE-706) Index File Format - Example for frequency file .frq is wrong

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
     [ http://issues.apache.org/jira/browse/LUCENE-706?page=all ]

Grant Ingersoll resolved LUCENE-706.
------------------------------------

    Resolution: Fixed

Applied.  Thanks Johan and Doron.

> Index File Format - Example for frequency file .frq is wrong
> ------------------------------------------------------------
>
>                 Key: LUCENE-706
>                 URL: http://issues.apache.org/jira/browse/LUCENE-706
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Website
>         Environment: not applicable
>            Reporter: Doron Cohen
>         Assigned To: Grant Ingersoll
>            Priority: Trivial
>         Attachments: file-format-frq-example.patch
>
>
> Reported by Johan Stuyts - http://www.nabble.com/Possible-documentation-error--p7012445.html -
> Frequency file example says:
>      For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
>          15, 22, 3
> It should be:
>      For example, the TermFreqs for a term which occurs once in document seven and three times in document eleven would be the following sequence of VInts:
>          15, 8, 3

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]