StandardTokenizerConstants in 2.3

classic Classic list List threaded Threaded
8 messages Options
adb
Reply | Threaded
Open this post in threaded view
|

StandardTokenizerConstants in 2.3

adb
I'm migrating from 2.1 to 2.3 and found that the public interface
StandardTokenizerConstants has gone.  It looks like the definitions have
disappeared inside the package private class StandardTokenizerImpl.

Was this intentional?  I was using these to determine the returns values from
Token.type().

Antony


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: StandardTokenizerConstants in 2.3

Michael McCandless-2

Unfortunately, we lost the StandardTokenizerConstants interface as
part of this:

     https://issues.apache.org/jira/browse/LUCENE-966

which was a speedup to StandardTokenizer by switching to JFlex instead
of JavaCC.

But, the constants that are used by StandardTokenizer are still
available as static ints in the StandardTokenizer class (ie, ALPHANUM,
APOSTROPHE, etc.).  Does that work?

Mike

Antony Bowesman wrote:

> I'm migrating from 2.1 to 2.3 and found that the public interface  
> StandardTokenizerConstants has gone.  It looks like the definitions  
> have disappeared inside the package private class  
> StandardTokenizerImpl.
>
> Was this intentional?  I was using these to determine the returns  
> values from Token.type().
>
> Antony
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

adb
Reply | Threaded
Open this post in threaded view
|

Re: StandardTokenizerConstants in 2.3

adb
> But, the constants that are used by StandardTokenizer are still
> available as static ints in the StandardTokenizer class (ie, ALPHANUM,
> APOSTROPHE, etc.).  Does that work?

Problem as mentioned below is that the StandardTokenizerImpl.java is package
private and even though the ints and string array are declared as public static,
they are not visible.

Antony


>
> Mike
>
> Antony Bowesman wrote:
>> I'm migrating from 2.1 to 2.3 and found that the public interface
>> StandardTokenizerConstants has gone.  It looks like the definitions
>> have disappeared inside the package private class StandardTokenizerImpl.
>>
>> Was this intentional?  I was using these to determine the returns
>> values from Token.type().
>>
>> Antony
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: StandardTokenizerConstants in 2.3

Michael McCandless-2

But, StandardTokenizer is public?  It "exports" those constants for you?

Mike

Antony Bowesman wrote:

>> But, the constants that are used by StandardTokenizer are still
>> available as static ints in the StandardTokenizer class (ie,  
>> ALPHANUM,
>> APOSTROPHE, etc.).  Does that work?
>
> Problem as mentioned below is that the StandardTokenizerImpl.java  
> is package private and even though the ints and string array are  
> declared as public static, they are not visible.
>
> Antony
>
>
>> Mike
>> Antony Bowesman wrote:
>>> I'm migrating from 2.1 to 2.3 and found that the public interface  
>>> StandardTokenizerConstants has gone.  It looks like the  
>>> definitions have disappeared inside the package private class  
>>> StandardTokenizerImpl.
>>>
>>> Was this intentional?  I was using these to determine the returns  
>>> values from Token.type().
>>>
>>> Antony
>>>
>>>
>>> --------------------------------------------------------------------
>>> -
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

adb
Reply | Threaded
Open this post in threaded view
|

Re: StandardTokenizerConstants in 2.3

adb
> But, StandardTokenizer is public?  It "exports" those constants for you?

Really?  Sorry, but I can't find them - in 2.3.1 sources, there are no
references to those statics.  Javadocs have no reference to them in
StandardTokenizer

http://lucene.apache.org/java/2_3_1/api/org/apache/lucene/analysis/standard/StandardTokenizer.html

and I can't see ALPHANUM in the Javadoc index.  Eclipse cannot resolve them.

Am I missing something?
Antony


>
> Mike
>
> Antony Bowesman wrote:
>>> But, the constants that are used by StandardTokenizer are still
>>> available as static ints in the StandardTokenizer class (ie, ALPHANUM,
>>> APOSTROPHE, etc.).  Does that work?
>>
>> Problem as mentioned below is that the StandardTokenizerImpl.java is
>> package private and even though the ints and string array are declared
>> as public static, they are not visible.
>>
>> Antony



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: StandardTokenizerConstants in 2.3

hossman

: > But, StandardTokenizer is public?  It "exports" those constants for you?
:
: Really?  Sorry, but I can't find them - in 2.3.1 sources, there are no
: references to those statics.  Javadocs have no reference to them in
: StandardTokenizer

I think Michael is forgetting that he re-added those constants to the
trunk after 2.3.1 was released...

https://issues.apache.org/jira/browse/LUCENE-1150



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: StandardTokenizerConstants in 2.3

Michael McCandless-2

Chris Hostetter wrote:

>
> : > But, StandardTokenizer is public?  It "exports" those constants  
> for you?
> :
> : Really?  Sorry, but I can't find them - in 2.3.1 sources, there  
> are no
> : references to those statics.  Javadocs have no reference to them in
> : StandardTokenizer
>
> I think Michael is forgetting that he re-added those constants to the
> trunk after 2.3.1 was released...
>
> https://issues.apache.org/jira/browse/LUCENE-1150

Woops!  I'm sorry Antony -- Hoss is correct.

I didn't realize this missed 2.3.  I'll backport this fix to 2.3  
branch so it'll be included when we release 2.3.2 (which I think we  
should do soon -- alot of little fixes have been backported).

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

adb
Reply | Threaded
Open this post in threaded view
|

Re: StandardTokenizerConstants in 2.3

adb
Thanks Mike/Hoss for the clarification.
Antony


Michael McCandless wrote:

>
> Chris Hostetter wrote:
>>
>> : > But, StandardTokenizer is public?  It "exports" those constants
>> for you?
>> :
>> : Really?  Sorry, but I can't find them - in 2.3.1 sources, there are no
>> : references to those statics.  Javadocs have no reference to them in
>> : StandardTokenizer
>>
>> I think Michael is forgetting that he re-added those constants to the
>> trunk after 2.3.1 was released...
>>
>> https://issues.apache.org/jira/browse/LUCENE-1150
>
> Woops!  I'm sorry Antony -- Hoss is correct.
>
> I didn't realize this missed 2.3.  I'll backport this fix to 2.3 branch
> so it'll be included when we release 2.3.2 (which I think we should do
> soon -- alot of little fixes have been backported).
>
> Mike
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]