[jira] [Created] (LUCENE-4234) Exception when FacetsCollector is used with ScoreFacetRequest

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (LUCENE-4234) Exception when FacetsCollector is used with ScoreFacetRequest

JIRA jira@apache.org
Gilad Barkai created LUCENE-4234:
------------------------------------

             Summary: Exception when FacetsCollector is used with ScoreFacetRequest
                 Key: LUCENE-4234
                 URL: https://issues.apache.org/jira/browse/LUCENE-4234
             Project: Lucene - Java
          Issue Type: Bug
          Components: modules/facet
    Affects Versions: 4.0-ALPHA, 3.6
            Reporter: Gilad Barkai
             Fix For: 3.6.1, 4.0


Aggregating facets with Lucene's Score using FacetsCollector results in an Exception (assertion when enabled).



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (LUCENE-4234) Exception when FacetsCollector is used with ScoreFacetRequest

JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gilad Barkai updated LUCENE-4234:
---------------------------------

    Attachment: LUCENE-4234-test.patch

A test revealing the bug.
               

> Exception when FacetsCollector is used with ScoreFacetRequest
> -------------------------------------------------------------
>
>                 Key: LUCENE-4234
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4234
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/facet
>    Affects Versions: 3.6, 4.0-ALPHA
>            Reporter: Gilad Barkai
>             Fix For: 3.6.1, 4.0
>
>         Attachments: LUCENE-4234-test.patch
>
>
> Aggregating facets with Lucene's Score using FacetsCollector results in an Exception (assertion when enabled).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (LUCENE-4234) Exception when FacetsCollector is used with ScoreFacetRequest

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gilad Barkai updated LUCENE-4234:
---------------------------------

    Attachment: LUCENE-4234-fix.patch

Proposed fix, the code was optimized for a capacity of 1000 documents, but BitSet was initizlized with the same value of 1000, causing .fastSet() to fail.

Initializing the scores array to size of 64, and initializing the bitset to .maxDoc() as in the non-score-keeping version.
               

> Exception when FacetsCollector is used with ScoreFacetRequest
> -------------------------------------------------------------
>
>                 Key: LUCENE-4234
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4234
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/facet
>    Affects Versions: 3.6, 4.0-ALPHA
>            Reporter: Gilad Barkai
>             Fix For: 3.6.1, 4.0
>
>         Attachments: LUCENE-4234-fix.patch, LUCENE-4234-test.patch
>
>
> Aggregating facets with Lucene's Score using FacetsCollector results in an Exception (assertion when enabled).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Assigned] (LUCENE-4234) Exception when FacetsCollector is used with ScoreFacetRequest

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shai Erera reassigned LUCENE-4234:
----------------------------------

    Assignee: Shai Erera
   

> Exception when FacetsCollector is used with ScoreFacetRequest
> -------------------------------------------------------------
>
>                 Key: LUCENE-4234
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4234
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/facet
>    Affects Versions: 3.6, 4.0-ALPHA
>            Reporter: Gilad Barkai
>            Assignee: Shai Erera
>             Fix For: 4.0, 5.0, 3.6.2
>
>         Attachments: LUCENE-4234-fix.patch, LUCENE-4234-test.patch
>
>
> Aggregating facets with Lucene's Score using FacetsCollector results in an Exception (assertion when enabled).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (LUCENE-4234) Exception when FacetsCollector is used with ScoreFacetRequest

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shai Erera updated LUCENE-4234:
-------------------------------

    Fix Version/s:     (was: 3.6.1)
                   3.6.2
                   5.0
   

> Exception when FacetsCollector is used with ScoreFacetRequest
> -------------------------------------------------------------
>
>                 Key: LUCENE-4234
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4234
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/facet
>    Affects Versions: 3.6, 4.0-ALPHA
>            Reporter: Gilad Barkai
>            Assignee: Shai Erera
>             Fix For: 4.0, 5.0, 3.6.2
>
>         Attachments: LUCENE-4234-fix.patch, LUCENE-4234-test.patch
>
>
> Aggregating facets with Lucene's Score using FacetsCollector results in an Exception (assertion when enabled).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-4234) Exception when FacetsCollector is used with ScoreFacetRequest

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420123#comment-13420123 ]

Shai Erera commented on LUCENE-4234:
------------------------------------

Good catch !

I have few comments about the patch:

# Can you merge them into one file?
# Can we use FixedBitSet? It's an improvement over OpenBitSet, without needing to use fastXYZ
# I think that we can initialize the scores array to maxDoc, and drop the resizing in collect -- collect is a hot method, we should do minimal work there. If someone want to keep scores for maxDoc documents, we should initialize the array up front.
               

> Exception when FacetsCollector is used with ScoreFacetRequest
> -------------------------------------------------------------
>
>                 Key: LUCENE-4234
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4234
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/facet
>    Affects Versions: 3.6, 4.0-ALPHA
>            Reporter: Gilad Barkai
>            Assignee: Shai Erera
>             Fix For: 4.0, 5.0, 3.6.2
>
>         Attachments: LUCENE-4234-fix.patch, LUCENE-4234-test.patch
>
>
> Aggregating facets with Lucene's Score using FacetsCollector results in an Exception (assertion when enabled).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (LUCENE-4234) Exception when FacetsCollector is used with ScoreFacetRequest

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gilad Barkai updated LUCENE-4234:
---------------------------------

    Attachment: LUCENE-4234.patch

One patch (to rule them all) instead of the two. No changes were made thus far.
               

> Exception when FacetsCollector is used with ScoreFacetRequest
> -------------------------------------------------------------
>
>                 Key: LUCENE-4234
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4234
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/facet
>    Affects Versions: 3.6, 4.0-ALPHA
>            Reporter: Gilad Barkai
>            Assignee: Shai Erera
>             Fix For: 4.0, 5.0, 3.6.2
>
>         Attachments: LUCENE-4234.patch
>
>
> Aggregating facets with Lucene's Score using FacetsCollector results in an Exception (assertion when enabled).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (LUCENE-4234) Exception when FacetsCollector is used with ScoreFacetRequest

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gilad Barkai updated LUCENE-4234:
---------------------------------

    Attachment:     (was: LUCENE-4234-fix.patch)
   

> Exception when FacetsCollector is used with ScoreFacetRequest
> -------------------------------------------------------------
>
>                 Key: LUCENE-4234
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4234
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/facet
>    Affects Versions: 3.6, 4.0-ALPHA
>            Reporter: Gilad Barkai
>            Assignee: Shai Erera
>             Fix For: 4.0, 5.0, 3.6.2
>
>         Attachments: LUCENE-4234.patch
>
>
> Aggregating facets with Lucene's Score using FacetsCollector results in an Exception (assertion when enabled).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (LUCENE-4234) Exception when FacetsCollector is used with ScoreFacetRequest

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gilad Barkai updated LUCENE-4234:
---------------------------------

    Attachment:     (was: LUCENE-4234-test.patch)
   

> Exception when FacetsCollector is used with ScoreFacetRequest
> -------------------------------------------------------------
>
>                 Key: LUCENE-4234
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4234
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/facet
>    Affects Versions: 3.6, 4.0-ALPHA
>            Reporter: Gilad Barkai
>            Assignee: Shai Erera
>             Fix For: 4.0, 5.0, 3.6.2
>
>         Attachments: LUCENE-4234.patch
>
>
> Aggregating facets with Lucene's Score using FacetsCollector results in an Exception (assertion when enabled).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-4234) Exception when FacetsCollector is used with ScoreFacetRequest

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420125#comment-13420125 ]

Gilad Barkai commented on LUCENE-4234:
--------------------------------------

Hi Shai, thank you for reviewing.

1. Merged
2. Using FBS is possible, though it's a larger change than the fix, perhaps handle this in a seperate issue?
3. IMHO that's a waste for a large index. While allocating a large array is faster, it takes a lot of memory - probably for no reason. Each query will take longer, but concurrent queries will not hurt the GC that much. If the index is large concurrent queries might hit the GC hard, perhaps even OOM ? I must admit I never measured the reallocation penalty.
               

> Exception when FacetsCollector is used with ScoreFacetRequest
> -------------------------------------------------------------
>
>                 Key: LUCENE-4234
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4234
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/facet
>    Affects Versions: 3.6, 4.0-ALPHA
>            Reporter: Gilad Barkai
>            Assignee: Shai Erera
>             Fix For: 4.0, 5.0, 3.6.2
>
>         Attachments: LUCENE-4234.patch
>
>
> Aggregating facets with Lucene's Score using FacetsCollector results in an Exception (assertion when enabled).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-4234) Exception when FacetsCollector is used with ScoreFacetRequest

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420126#comment-13420126 ]

Shai Erera commented on LUCENE-4234:
------------------------------------

Ok I understand the code better now -- the scores array keeps scores only for docs that were visited by Scorer. Therefore it doesn't need to be of size maxDoc, but rather close to how many docs were visited.

Makes sense I guess for queries that hit a low percentage of the index. I'm not worried about the re-allocation work at all ... I didn't dig deep into the code first, and thought that we're keeping an entry per document in the index, whether it's scored or not.

The current code is good then.

I'll review the new patch now.
               

> Exception when FacetsCollector is used with ScoreFacetRequest
> -------------------------------------------------------------
>
>                 Key: LUCENE-4234
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4234
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/facet
>    Affects Versions: 3.6, 4.0-ALPHA
>            Reporter: Gilad Barkai
>            Assignee: Shai Erera
>             Fix For: 4.0, 5.0, 3.6.2
>
>         Attachments: LUCENE-4234.patch
>
>
> Aggregating facets with Lucene's Score using FacetsCollector results in an Exception (assertion when enabled).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (LUCENE-4234) Exception when FacetsCollector is used with ScoreFacetRequest

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shai Erera updated LUCENE-4234:
-------------------------------

    Attachment: LUCENE-4234.patch

Updated patch file:

* Use FixedBitSet - the change was trivial
* Add a comment to new float[64] explaining why we do it (I know it got me confused once ...)
* Added CHANGES entry

All tests pass. I'll commit it by tomorrow.
               

> Exception when FacetsCollector is used with ScoreFacetRequest
> -------------------------------------------------------------
>
>                 Key: LUCENE-4234
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4234
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/facet
>    Affects Versions: 3.6, 4.0-ALPHA
>            Reporter: Gilad Barkai
>            Assignee: Shai Erera
>             Fix For: 4.0, 5.0, 3.6.2
>
>         Attachments: LUCENE-4234.patch, LUCENE-4234.patch
>
>
> Aggregating facets with Lucene's Score using FacetsCollector results in an Exception (assertion when enabled).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-4234) Exception when FacetsCollector is used with ScoreFacetRequest

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420144#comment-13420144 ]

Gilad Barkai commented on LUCENE-4234:
--------------------------------------

Thank you for looking into this and making the appropriate changes.
Patch looks good.
               

> Exception when FacetsCollector is used with ScoreFacetRequest
> -------------------------------------------------------------
>
>                 Key: LUCENE-4234
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4234
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/facet
>    Affects Versions: 3.6, 4.0-ALPHA
>            Reporter: Gilad Barkai
>            Assignee: Shai Erera
>             Fix For: 4.0, 5.0, 3.6.2
>
>         Attachments: LUCENE-4234.patch, LUCENE-4234.patch
>
>
> Aggregating facets with Lucene's Score using FacetsCollector results in an Exception (assertion when enabled).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-4234) Exception when FacetsCollector is used with ScoreFacetRequest

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420556#comment-13420556 ]

Shai Erera commented on LUCENE-4234:
------------------------------------

Committed to trunk rev 1364576. Porting now to 4x an 3.6.2.
               

> Exception when FacetsCollector is used with ScoreFacetRequest
> -------------------------------------------------------------
>
>                 Key: LUCENE-4234
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4234
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: modules/facet
>    Affects Versions: 3.6, 4.0-ALPHA
>            Reporter: Gilad Barkai
>            Assignee: Shai Erera
>             Fix For: 4.0, 5.0, 3.6.2
>
>         Attachments: LUCENE-4234.patch, LUCENE-4234.patch
>
>
> Aggregating facets with Lucene's Score using FacetsCollector results in an Exception (assertion when enabled).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Resolved] (LUCENE-4234) Exception when FacetsCollector is used with ScoreFacetRequest

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shai Erera resolved LUCENE-4234.
--------------------------------

       Resolution: Fixed
    Lucene Fields: New,Patch Available  (was: New)

Committed r1364582 (4x) and 1364586 (3.6.2).

Thanks Gilad !
               

> Exception when FacetsCollector is used with ScoreFacetRequest
> -------------------------------------------------------------
>
>                 Key: LUCENE-4234
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4234
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: modules/facet
>    Affects Versions: 3.6, 4.0-ALPHA
>            Reporter: Gilad Barkai
>            Assignee: Shai Erera
>             Fix For: 4.0, 5.0, 3.6.2
>
>         Attachments: LUCENE-4234.patch, LUCENE-4234.patch
>
>
> Aggregating facets with Lucene's Score using FacetsCollector results in an Exception (assertion when enabled).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]