[jira] [Commented] (LUCENE-4656) Fix EmptyTokenizer

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-4656) Fix EmptyTokenizer

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-4656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13543111#comment-13543111 ]

Robert Muir commented on LUCENE-4656:
-------------------------------------

Slightly related to the BaseToken changes, i think its confusing how we use output.length (from the String[]) also as the number of expected tokens.

we could clear this up with something like:
{noformat}
@@ -114,21 +114,32 @@
   public static void assertTokenStreamContents(...
     assertNotNull(output);
+    final int numExpected = output.length;
{noformat}

and then use this in the for loop and such.

additionally i've often sent the wrong number of parameters when writing tests because you are passing huge parallel arrays.
so something like this could save some trouble:

{noformat}
     TypeAttribute typeAtt = null;
     if (types != null) {
       assertTrue("has no TypeAttribute", ts.hasAttribute(TypeAttribute.class));
       typeAtt = ts.getAttribute(TypeAttribute.class);
+      assertEquals("wrong number of types", numExpected, types.length);
     }
{noformat}

We don't have to do these changes here. it just reminded me of it looking at this stuff.

               

> Fix EmptyTokenizer
> ------------------
>
>                 Key: LUCENE-4656
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4656
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: modules/analysis
>            Reporter: Adrien Grand
>            Assignee: Uwe Schindler
>            Priority: Trivial
>         Attachments: LUCENE-4656_bttc.patch, LUCENE-4656-IW-bug.patch, LUCENE-4656-IW-fix.patch, LUCENE-4656-IW-fix.patch, LUCENE-4656.patch, LUCENE-4656.patch, LUCENE-4656.patch, LUCENE-4656.patch, LUCENE-4656.patch
>
>
> TestRandomChains can fail because EmptyTokenizer doesn't have a CharTermAttribute and doesn't compute the end offset (if the offset attribute was added by a filter).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]