Porting Analyzer from ver 4.8.1 to ver 6.4.1

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Porting Analyzer from ver 4.8.1 to ver 6.4.1

Vincenzo D'Amore
Hi all,

I'm porting few classes from 4.8.1 to the newer version of Lucene. I can't
understand how to convert this code, I hope you can help me:

  public  Analyzer creaAnalyzer() {
  return new Analyzer() {

  @Override
protected TokenStreamComponents createComponents(String fieldName, Reader
arg1) {
Tokenizer tokenizer = new WhitespaceTokenizer(Version.LUCENE_CURRENT, arg1);
TokenStream result = new LowerCaseFilter(Version.LUCENE_CURRENT, tokenizer);
  boolean ignorecase=true;
  result = new SynonymFilter(result, creazioneDizionario(), ignorecase);
  return new TokenStreamComponents(tokenizer,result);
  }
  };
  }

My proposal is this:

  public  Analyzer creaAnalyzer() {
  return new Analyzer() {

  @Override
+ protected TokenStreamComponents createComponents(String fieldName) {
+ Tokenizer tokenizer = new WhitespaceTokenizer();
+                               tokenizer.setReader(reader);
+  TokenStream result = new LowerCaseFilter(tokenizer);

   boolean ignorecase=true;
   result = new SynonymFilter(result, creazioneDizionario(), ignorecase);
  return new TokenStreamComponents(tokenizer,result);
  }
  };
  }

but I can't understand how to pass the "reader" to the analyzer and then to
the WhitespaceTokenizer.

Am I wrong? What do you suggest me?

Thanks in advance for your help.
Vincenzo
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Porting Analyzer from ver 4.8.1 to ver 6.4.1

Uwe Schindler
Hi,

Remove the setReader()? Tokenizers do not need a Reader when they are constructed (as they are for reuse). Once created the same instance is used over and over, so it is initialized with a Reader before each use. This was never different in the past, the was just the additional hurdle to handle the difference between first call and followup calls. No its quite easy separated: Create it in one step without a reader and then reuse it as often as you like by setting a Reader. This is done inside the final Analyzer logic that is hidden to you!

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Vincenzo D'Amore [mailto:[hidden email]]
> Sent: Thursday, February 16, 2017 6:56 PM
> To: [hidden email]
> Subject: Porting Analyzer from ver 4.8.1 to ver 6.4.1
>
> Hi all,
>
> I'm porting few classes from 4.8.1 to the newer version of Lucene. I can't
> understand how to convert this code, I hope you can help me:
>
>   public  Analyzer creaAnalyzer() {
>   return new Analyzer() {
>
>   @Override
> protected TokenStreamComponents createComponents(String fieldName,
> Reader
> arg1) {
> Tokenizer tokenizer = new WhitespaceTokenizer(Version.LUCENE_CURRENT,
> arg1);
> TokenStream result = new LowerCaseFilter(Version.LUCENE_CURRENT,
> tokenizer);
>   boolean ignorecase=true;
>   result = new SynonymFilter(result, creazioneDizionario(), ignorecase);
>   return new TokenStreamComponents(tokenizer,result);
>   }
>   };
>   }
>
> My proposal is this:
>
>   public  Analyzer creaAnalyzer() {
>   return new Analyzer() {
>
>   @Override
> + protected TokenStreamComponents createComponents(String fieldName) {
> + Tokenizer tokenizer = new WhitespaceTokenizer();
> +                               tokenizer.setReader(reader);
> +  TokenStream result = new LowerCaseFilter(tokenizer);
>
>    boolean ignorecase=true;
>    result = new SynonymFilter(result, creazioneDizionario(), ignorecase);
>   return new TokenStreamComponents(tokenizer,result);
>   }
>   };
>   }
>
> but I can't understand how to pass the "reader" to the analyzer and then to
> the WhitespaceTokenizer.
>
> Am I wrong? What do you suggest me?
>
> Thanks in advance for your help.
> Vincenzo


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Porting Analyzer from ver 4.8.1 to ver 6.4.1

Vincenzo D'Amore
Thank you, works like a charm.
Loading...