[jira] [Commented] (LUCENE-7968) AnalyzingSuggester's comparator compares incorrectly

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-7968) AnalyzingSuggester's comparator compares incorrectly

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-7968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16163940#comment-16163940 ]

Robert Muir commented on LUCENE-7968:
-------------------------------------

Here's an assert that can currently make tests fail. But I want to test that the tie-break case really works, not just that stuff is in-bounds.

{code}
diff --git a/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingSuggester.java b/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingSuggester.java
index 9c6a624..2fa3569 100644
--- a/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingSuggester.java
+++ b/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingSuggester.java
@@ -392,6 +392,8 @@ public class AnalyzingSuggester extends Lookup implements Accountable {
         scratchB.offset = readerB.getPosition();
         scratchA.length = a.length - scratchA.offset;
         scratchB.length = b.length - scratchB.offset;
+        assert scratchA.isValid();
+        assert scratchB.isValid();
       }
   
       return scratchA.compareTo(scratchB);
{code}

{noformat}
   [junit4] ERROR   0.31s J2 | AnalyzingSuggesterTest.testRandom <<<
   [junit4]    > Throwable #1: java.lang.IllegalStateException: length is negative: -1507
   [junit4]    > at __randomizedtesting.SeedInfo.seed([D27F69F0C3A46E64:A0334CFF72C4D817]:0)
   [junit4]    > at org.apache.lucene.util.BytesRef.isValid(BytesRef.java:222)
   [junit4]    > at org.apache.lucene.search.suggest.analyzing.AnalyzingSuggester$AnalyzingComparator.compare(AnalyzingSuggester.java:395)
   [junit4]    > at org.apache.lucene.search.suggest.analyzing.AnalyzingSuggester$AnalyzingComparator.compare(AnalyzingSuggester.java:339)
   [junit4]    > at org.apache.lucene.util.BytesRefArray$1.comparePivot(BytesRefArray.java:157)
   [junit4]    > at org.apache.lucene.util.IntroSorter.quicksort(IntroSorter.java:66)
   [junit4]    > at org.apache.lucene.util.IntroSorter.quicksort(IntroSorter.java:82)
   [junit4]    > at org.apache.lucene.util.IntroSorter.sort(IntroSorter.java:36)
   [junit4]    > at org.apache.lucene.util.BytesRefArray.sort(BytesRefArray.java:166)
   [junit4]    > at org.apache.lucene.util.BytesRefArray.iterator(BytesRefArray.java:196)
   [junit4]    > at org.apache.lucene.util.OfflineSorter$SortPartitionTask.call(OfflineSorter.java:606)
   [junit4]    > at org.apache.lucene.util.OfflineSorter$SortPartitionTask.call(OfflineSorter.java:588)
   [junit4]    > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]    > at org.apache.lucene.util.SameThreadExecutorService.execute(SameThreadExecutorService.java:34)
   [junit4]    > at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)
   [junit4]    > at org.apache.lucene.util.OfflineSorter.sort(OfflineSorter.java:285)
   [junit4]    > at org.apache.lucene.search.suggest.analyzing.AnalyzingSuggester.build(AnalyzingSuggester.java:491)
   [junit4]    > at org.apache.lucene.search.suggest.analyzing.AnalyzingSuggesterTest.testRandom(AnalyzingSuggesterTest.java:787)
   [junit4]    > at java.lang.Thread.run(Thread.java:745)
{noformat}

> AnalyzingSuggester's comparator compares incorrectly
> ----------------------------------------------------
>
>                 Key: LUCENE-7968
>                 URL: https://issues.apache.org/jira/browse/LUCENE-7968
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Robert Muir
>
> Found by LUCENE-7966, but we should fix here separate.
> Currently the tie-break case of this comparator is buggy when {{hasPayloads=false}}, as it assigns *negative* lengths to the BytesRefs being compared. The current BytesRef.compareTo simply silently returns false in this case, hiding the bug.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]