analysis back compat break

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

analysis back compat break

Yonik Seeley-2-2
Unfortunately it doesn't look like analysis is actually back compatible.

Consider the following token filter snippet that used to work fine:

  public Token next(Token target) throws IOException {;
    Token peek =;

What I'm seeing now is that the target and peek tokens are both the same.
This is because the "input" stream supports incrementToken and thus
incrementToken() is called first, changing "target".

Relevant code from TokenStream:
  public Token next() throws IOException {
    if (tokenWrapper == null)
      throw new UnsupportedOperationException("This TokenStream only
supports the new Attributes API.");

    if (supportedMethods.hasIncrementToken) {
      return incrementToken() ? ((Token) tokenWrapper.delegate.clone()) : null;


To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]