[jira] Created: (LUCENE-2806) ArrayIndexOutOfBoundsException thrown from TermScorer.score while doing a regular index search.

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (LUCENE-2806) ArrayIndexOutOfBoundsException thrown from TermScorer.score while doing a regular index search.

Hudson (Jira)
ArrayIndexOutOfBoundsException thrown from TermScorer.score while doing a regular index search.
-----------------------------------------------------------------------------------------------

                 Key: LUCENE-2806
                 URL: https://issues.apache.org/jira/browse/LUCENE-2806
             Project: Lucene - Java
          Issue Type: Bug
          Components: Search
    Affects Versions: 3.0.3
         Environment: Ubuntu linux
            Reporter: Soujanya


java.lang.ArrayIndexOutOfBoundsException: 51064
        org.apache.lucene.search.TermScorer.score(TermScorer.java:130)
        org.apache.lucene.search.TopScoreDocCollector$InOrderTopScoreDocCollector.collect(TopScoreDocCollector.java:47)
        org.apache.lucene.search.TermScorer.score(TermScorer.java:78)
        org.apache.lucene.search.TermScorer.score(TermScorer.java:70)
        org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:212)
        org.apache.lucene.search.Searcher.search(Searcher.java:67)


--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2806) ArrayIndexOutOfBoundsException thrown from TermScorer.score while doing a regular index search.

Hudson (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12968643#action_12968643 ]

Shai Erera commented on LUCENE-2806:
------------------------------------

Can you please post a patch with a short test case that reproduces the bug? If not, can you please describe the environment: index size, num docs, query used etc.

From TermScorer's code, the AIOOBE can be thrown if the 'freq' is larger than the scoreCache array size, however there is an if validating that freq is smaller than the array size, so I don't get it ...

> ArrayIndexOutOfBoundsException thrown from TermScorer.score while doing a regular index search.
> -----------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-2806
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2806
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Search
>    Affects Versions: 3.0.3
>         Environment: Ubuntu linux
>            Reporter: Soujanya
>
> java.lang.ArrayIndexOutOfBoundsException: 51064
> org.apache.lucene.search.TermScorer.score(TermScorer.java:130)
> org.apache.lucene.search.TopScoreDocCollector$InOrderTopScoreDocCollector.collect(TopScoreDocCollector.java:47)
> org.apache.lucene.search.TermScorer.score(TermScorer.java:78)
> org.apache.lucene.search.TermScorer.score(TermScorer.java:70)
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:212)
> org.apache.lucene.search.Searcher.search(Searcher.java:67)

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2806) ArrayIndexOutOfBoundsException thrown from TermScorer.score while doing a regular index search.

Hudson (Jira)
In reply to this post by Hudson (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12968647#action_12968647 ]

Soujanya commented on LUCENE-2806:
----------------------------------

I used SimpleAnalyzer to build the index. There are totally 240,000+
documents (multiple fielded documents).
I use SimpleAnalyzer to generate the query. The query has two fields to
search like this: field1:searchstring* AND field2:searchstring2. The first
one has a regular expression '*' at the end of the query string. To match
obtain matches when the query string is not entered completely. The query
string is a single word appended with a '*', say like 'field1:glamour* AND
type:document'.
I run this query on the index and it throws this error.

Could you please tell me what 'freq' is? Is the frequency of a word or is it
an array?

Thanks,
Soujanya




> ArrayIndexOutOfBoundsException thrown from TermScorer.score while doing a regular index search.
> -----------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-2806
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2806
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Search
>    Affects Versions: 3.0.3
>         Environment: Ubuntu linux
>            Reporter: Soujanya
>
> java.lang.ArrayIndexOutOfBoundsException: 51064
> org.apache.lucene.search.TermScorer.score(TermScorer.java:130)
> org.apache.lucene.search.TopScoreDocCollector$InOrderTopScoreDocCollector.collect(TopScoreDocCollector.java:47)
> org.apache.lucene.search.TermScorer.score(TermScorer.java:78)
> org.apache.lucene.search.TermScorer.score(TermScorer.java:70)
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:212)
> org.apache.lucene.search.Searcher.search(Searcher.java:67)

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2806) ArrayIndexOutOfBoundsException thrown from TermScorer.score while doing a regular index search.

Hudson (Jira)
In reply to this post by Hudson (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12968654#action_12968654 ]

Shai Erera commented on LUCENE-2806:
------------------------------------

Ok -- I looked at 3x source code, but I see you marked the issue as 3.0.3 and that TermScorer's lines are different between 3x and 3.0.3. The offending line looks like:

{code}
    return norms == null ? raw : raw * SIM_NORM_DECODER[norms[doc] & 0xFF]; // normalize for field
{code}

And it can be thrown by either invalid access to the 'norms' array, or the SIM_NORM_DECODER array. Are you using a custom Similarity maybe? I think it's less likely that the invalid access is caused from the 'norms' array, as it is allocated in the size of the index (one norm per document).

Are you able to rebuild the index? If so, I'd suggest to rebuild it and add all content as ANALYZED_NO_NORMS to disable norms. If the AIOOBE still happens, then we know it's from the SIM_NORM_DECODER, otherwise it's from the norms[] array.

Also, does it happen in other queries too? Maybe to remove some noise, remove the *field:word** part and only keep the type:document part? Trying to minimize the query down to few select terms to ease on the debugging.

Can you perhaps post the index here?

> ArrayIndexOutOfBoundsException thrown from TermScorer.score while doing a regular index search.
> -----------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-2806
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2806
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Search
>    Affects Versions: 3.0.3
>         Environment: Ubuntu linux
>            Reporter: Soujanya
>
> java.lang.ArrayIndexOutOfBoundsException: 51064
> org.apache.lucene.search.TermScorer.score(TermScorer.java:130)
> org.apache.lucene.search.TopScoreDocCollector$InOrderTopScoreDocCollector.collect(TopScoreDocCollector.java:47)
> org.apache.lucene.search.TermScorer.score(TermScorer.java:78)
> org.apache.lucene.search.TermScorer.score(TermScorer.java:70)
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:212)
> org.apache.lucene.search.Searcher.search(Searcher.java:67)

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]