CustomAnalyzer.Builder builder()

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

CustomAnalyzer.Builder builder()

leoperezpulido
Hi,


The documentation for the CustomAnalyzer Analyzer is very specific, one can create an instance of a custom analyzer in three different ways.


Using the default CustomAnalizer.builder(...), is supposed to load all resources from the classpath, and it works ok inside the IDE. But when I deploy the app I get the exception:


java.io.IOException: Resource not found:

at org.apache.lucene.analysis.util.ClasspathResourceLoader.openResource(ClasspathResourceLoader.java:67)
at org.apache.lucene.analysis.util.AbstractAnalysisFactory.getLines(AbstractAnalysisFactory.java:254)
at org.apache.lucene.analysis.util.AbstractAnalysisFactory.getWordSet(AbstractAnalysisFactory.java:243)
at org.apache.lucene.analysis.core.StopFilterFactory.inform(StopFilterFactory.java:96)
at org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.applyResourceLoader(CustomAnalyzer.java:413)
at org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.addTokenFilter(CustomAnalyzer.java:308)
at org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.addTokenFilter(CustomAnalyzer.java:299)
at mysearchapp.Page.<init>(Page.java:36)
at mysearchapp.MySearchApp.main(MySearchApp.java:53)


When I check the classpath for the resources inside MANIFEST.MF everything seems to be in order.



Leonardo.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: CustomAnalyzer.Builder builder()

Uwe Schindler
Hi,

the resources must be setup inside the JAR file in the same way like in classpath of your IDE. As you did not give any examples and also removed the resource name from your stack trace so we cannot figure out what you are doing!

The default builder uses the context classloader of current thread. This means that your application server (runtime environment) has to be setup correctly. In most cases for production use this is not ideal, as you have no control where the classes are loaded from and leave it to the externally defined properties. To fully define where the reosurces should be loaded from, construct your own ClasspathResourceLoader, e.g. by giving your own class as parameter. Then all resources are loaded directly from you Class's package inside the JAR file.

You say that you have the resources below META-INF: If you do this, the path you pass to load the resources must also start with /META-INF! Resources in Java have nothing to do with the META-INF folder. They are loaded like classes from the root folder of the JAR, so the path must be relative to JAR file's root.

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Leonardo Pérez Pulido [mailto:[hidden email]]
> Sent: Thursday, March 30, 2017 12:34 AM
> To: [hidden email]
> Subject: CustomAnalyzer.Builder builder()
>
> Hi,
>
>
> The documentation for the CustomAnalyzer Analyzer is very specific, one can
> create an instance of a custom analyzer in three different ways.
>
>
> Using the default CustomAnalizer.builder(...), is supposed to load all
> resources from the classpath, and it works ok inside the IDE. But when I
> deploy the app I get the exception:
>
>
> java.io.IOException: Resource not found:
>
> at
> org.apache.lucene.analysis.util.ClasspathResourceLoader.openResource(Clas
> spathResourceLoader.java:67)
> at
> org.apache.lucene.analysis.util.AbstractAnalysisFactory.getLines(AbstractAna
> lysisFactory.java:254)
> at
> org.apache.lucene.analysis.util.AbstractAnalysisFactory.getWordSet(Abstract
> AnalysisFactory.java:243)
> at
> org.apache.lucene.analysis.core.StopFilterFactory.inform(StopFilterFactory.ja
> va:96)
> at
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.applyResourceL
> oader(CustomAnalyzer.java:413)
> at
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.addTokenFilter(
> CustomAnalyzer.java:308)
> at
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.addTokenFilter(
> CustomAnalyzer.java:299)
> at mysearchapp.Page.<init>(Page.java:36)
> at mysearchapp.MySearchApp.main(MySearchApp.java:53)
>
>
> When I check the classpath for the resources inside MANIFEST.MF everything
> seems to be in order.
>
>
>
> Leonardo.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: CustomAnalyzer.Builder builder()

leoperezpulido
Hi,


Yes you are right, using the ClasspathResourceLoader class did solved the issue, passing my on class as parameter:


ClasspathResourceLoader resourceLoader = new
          ClasspathResourceLoader(MySearchApp.class);
Analyzer analyzer = CustomAnalyzer.builder(resourceLoader)
  .withTokenizer(StandardTokenizerFactory.class)
  .addTokenFilter(StandardFilterFactory.class)
  .addTokenFilter(LowerCaseFilterFactory.class)
  .addTokenFilter(ASCIIFoldingFilterFactory.class)
  .addTokenFilter(StopFilterFactory.class,
    "ignoreCase", "true",
    "words", "",
    "format", "wordset")
  .build();


As you can see, what I am trying to achieve is to add ASCIIFolfingFilterFactory to a custom analyzer chain.

Thanks very much!

Leonardo.

________________________________
From: Uwe Schindler <[hidden email]>
Sent: 30 March 2017 02:58 AM
To: [hidden email]
Subject: RE: CustomAnalyzer.Builder builder()

Hi,

the resources must be setup inside the JAR file in the same way like in classpath of your IDE. As you did not give any examples and also removed the resource name from your stack trace so we cannot figure out what you are doing!

The default builder uses the context classloader of current thread. This means that your application server (runtime environment) has to be setup correctly. In most cases for production use this is not ideal, as you have no control where the classes are loaded from and leave it to the externally defined properties. To fully define where the reosurces should be loaded from, construct your own ClasspathResourceLoader, e.g. by giving your own class as parameter. Then all resources are loaded directly from you Class's package inside the JAR file.

You say that you have the resources below META-INF: If you do this, the path you pass to load the resources must also start with /META-INF! Resources in Java have nothing to do with the META-INF folder. They are loaded like classes from the root folder of the JAR, so the path must be relative to JAR file's root.

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Leonardo Pérez Pulido [mailto:[hidden email]]
> Sent: Thursday, March 30, 2017 12:34 AM
> To: [hidden email]
> Subject: CustomAnalyzer.Builder builder()
>
> Hi,
>
>
> The documentation for the CustomAnalyzer Analyzer is very specific, one can
> create an instance of a custom analyzer in three different ways.
>
>
> Using the default CustomAnalizer.builder(...), is supposed to load all
> resources from the classpath, and it works ok inside the IDE. But when I
> deploy the app I get the exception:
>
>
> java.io.IOException: Resource not found:
>
> at
> org.apache.lucene.analysis.util.ClasspathResourceLoader.openResource(Clas
> spathResourceLoader.java:67)
> at
> org.apache.lucene.analysis.util.AbstractAnalysisFactory.getLines(AbstractAna
> lysisFactory.java:254)
> at
> org.apache.lucene.analysis.util.AbstractAnalysisFactory.getWordSet(Abstract
> AnalysisFactory.java:243)
> at
> org.apache.lucene.analysis.core.StopFilterFactory.inform(StopFilterFactory.ja
> va:96)
> at
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.applyResourceL
> oader(CustomAnalyzer.java:413)
> at
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.addTokenFilter(
> CustomAnalyzer.java:308)
> at
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.addTokenFilter(
> CustomAnalyzer.java:299)
> at mysearchapp.Page.<init>(Page.java:36)
> at mysearchapp.MySearchApp.main(MySearchApp.java:53)
>
>
> When I check the classpath for the resources inside MANIFEST.MF everything
> seems to be in order.
>
>
>
> Leonardo.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: CustomAnalyzer.Builder builder()

Uwe Schindler
Hi,

I am still a bit confused why you use an empty file name! Is this just copypasted here for privacy reasons wthout filename, or is there really no file name? This would explain why it may not work with the defaults.

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Leonardo Pérez Pulido [mailto:[hidden email]]
> Sent: Thursday, March 30, 2017 2:30 PM
> To: [hidden email]
> Subject: Re: CustomAnalyzer.Builder builder()
>
> Hi,
>
>
> Yes you are right, using the ClasspathResourceLoader class did solved the
> issue, passing my on class as parameter:
>
>
> ClasspathResourceLoader resourceLoader = new
>           ClasspathResourceLoader(MySearchApp.class);
> Analyzer analyzer = CustomAnalyzer.builder(resourceLoader)
>   .withTokenizer(StandardTokenizerFactory.class)
>   .addTokenFilter(StandardFilterFactory.class)
>   .addTokenFilter(LowerCaseFilterFactory.class)
>   .addTokenFilter(ASCIIFoldingFilterFactory.class)
>   .addTokenFilter(StopFilterFactory.class,
>     "ignoreCase", "true",
>     "words", "",
>     "format", "wordset")
>   .build();
>
>
> As you can see, what I am trying to achieve is to add ASCIIFolfingFilterFactory
> to a custom analyzer chain.
>
> Thanks very much!
>
> Leonardo.
>
> ________________________________
> From: Uwe Schindler <[hidden email]>
> Sent: 30 March 2017 02:58 AM
> To: [hidden email]
> Subject: RE: CustomAnalyzer.Builder builder()
>
> Hi,
>
> the resources must be setup inside the JAR file in the same way like in
> classpath of your IDE. As you did not give any examples and also removed
> the resource name from your stack trace so we cannot figure out what you
> are doing!
>
> The default builder uses the context classloader of current thread. This
> means that your application server (runtime environment) has to be setup
> correctly. In most cases for production use this is not ideal, as you have no
> control where the classes are loaded from and leave it to the externally
> defined properties. To fully define where the reosurces should be loaded
> from, construct your own ClasspathResourceLoader, e.g. by giving your own
> class as parameter. Then all resources are loaded directly from you Class's
> package inside the JAR file.
>
> You say that you have the resources below META-INF: If you do this, the path
> you pass to load the resources must also start with /META-INF! Resources in
> Java have nothing to do with the META-INF folder. They are loaded like
> classes from the root folder of the JAR, so the path must be relative to JAR
> file's root.
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> http://www.thetaphi.de
> eMail: [hidden email]
>
> > -----Original Message-----
> > From: Leonardo Pérez Pulido [mailto:[hidden email]]
> > Sent: Thursday, March 30, 2017 12:34 AM
> > To: [hidden email]
> > Subject: CustomAnalyzer.Builder builder()
> >
> > Hi,
> >
> >
> > The documentation for the CustomAnalyzer Analyzer is very specific, one
> can
> > create an instance of a custom analyzer in three different ways.
> >
> >
> > Using the default CustomAnalizer.builder(...), is supposed to load all
> > resources from the classpath, and it works ok inside the IDE. But when I
> > deploy the app I get the exception:
> >
> >
> > java.io.IOException: Resource not found:
> >
> > at
> >
> org.apache.lucene.analysis.util.ClasspathResourceLoader.openResource(Clas
> > spathResourceLoader.java:67)
> > at
> >
> org.apache.lucene.analysis.util.AbstractAnalysisFactory.getLines(AbstractAna
> > lysisFactory.java:254)
> > at
> >
> org.apache.lucene.analysis.util.AbstractAnalysisFactory.getWordSet(Abstract
> > AnalysisFactory.java:243)
> > at
> >
> org.apache.lucene.analysis.core.StopFilterFactory.inform(StopFilterFactory.ja
> > va:96)
> > at
> >
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.applyResourceL
> > oader(CustomAnalyzer.java:413)
> > at
> >
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.addTokenFilter(
> > CustomAnalyzer.java:308)
> > at
> >
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.addTokenFilter(
> > CustomAnalyzer.java:299)
> > at mysearchapp.Page.<init>(Page.java:36)
> > at mysearchapp.MySearchApp.main(MySearchApp.java:53)
> >
> >
> > When I check the classpath for the resources inside MANIFEST.MF
> everything
> > seems to be in order.
> >
> >
> >
> > Leonardo.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: CustomAnalyzer.Builder builder()

leoperezpulido
Hi,


I do not know to what file name do you refer, can you please be a little more specific? Did you mean a stopwords file name?


Currently I am developing  an application which do not have a name yet, this is why you see the class MySearchApp, which is a test source file on which I am testing Lucene features, in this case the CustomAnalyzer.Builder class to build a custom analyzer.


If you refer to the empty argument on StopFilterFactory.class, which we can use to pass a stopwords file to parse, the documentation says that all arguments are optional and in case of an empty stopwords file name, then Lucene will use StopAnalyzer.ENGLISH_STOP_WORDS_SET by default. Is is true?


There is no secrets here, I am only trying to learn Lucene.


Leonardo.

________________________________
From: Uwe Schindler <[hidden email]>
Sent: 30 March 2017 08:48:13
To: [hidden email]
Subject: RE: CustomAnalyzer.Builder builder()

Hi,

I am still a bit confused why you use an empty file name! Is this just copypasted here for privacy reasons wthout filename, or is there really no file name? This would explain why it may not work with the defaults.

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Leonardo Pérez Pulido [mailto:[hidden email]]
> Sent: Thursday, March 30, 2017 2:30 PM
> To: [hidden email]
> Subject: Re: CustomAnalyzer.Builder builder()
>
> Hi,
>
>
> Yes you are right, using the ClasspathResourceLoader class did solved the
> issue, passing my on class as parameter:
>
>
> ClasspathResourceLoader resourceLoader = new
>           ClasspathResourceLoader(MySearchApp.class);
> Analyzer analyzer = CustomAnalyzer.builder(resourceLoader)
>   .withTokenizer(StandardTokenizerFactory.class)
>   .addTokenFilter(StandardFilterFactory.class)
>   .addTokenFilter(LowerCaseFilterFactory.class)
>   .addTokenFilter(ASCIIFoldingFilterFactory.class)
>   .addTokenFilter(StopFilterFactory.class,
>     "ignoreCase", "true",
>     "words", "",
>     "format", "wordset")
>   .build();
>
>
> As you can see, what I am trying to achieve is to add ASCIIFolfingFilterFactory
> to a custom analyzer chain.
>
> Thanks very much!
>
> Leonardo.
>
> ________________________________
> From: Uwe Schindler <[hidden email]>
> Sent: 30 March 2017 02:58 AM
> To: [hidden email]
> Subject: RE: CustomAnalyzer.Builder builder()
>
> Hi,
>
> the resources must be setup inside the JAR file in the same way like in
> classpath of your IDE. As you did not give any examples and also removed
> the resource name from your stack trace so we cannot figure out what you
> are doing!
>
> The default builder uses the context classloader of current thread. This
> means that your application server (runtime environment) has to be setup
> correctly. In most cases for production use this is not ideal, as you have no
> control where the classes are loaded from and leave it to the externally
> defined properties. To fully define where the reosurces should be loaded
> from, construct your own ClasspathResourceLoader, e.g. by giving your own
> class as parameter. Then all resources are loaded directly from you Class's
> package inside the JAR file.
>
> You say that you have the resources below META-INF: If you do this, the path
> you pass to load the resources must also start with /META-INF! Resources in
> Java have nothing to do with the META-INF folder. They are loaded like
> classes from the root folder of the JAR, so the path must be relative to JAR
> file's root.
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> http://www.thetaphi.de
> eMail: [hidden email]
>
> > -----Original Message-----
> > From: Leonardo Pérez Pulido [mailto:[hidden email]]
> > Sent: Thursday, March 30, 2017 12:34 AM
> > To: [hidden email]
> > Subject: CustomAnalyzer.Builder builder()
> >
> > Hi,
> >
> >
> > The documentation for the CustomAnalyzer Analyzer is very specific, one
> can
> > create an instance of a custom analyzer in three different ways.
> >
> >
> > Using the default CustomAnalizer.builder(...), is supposed to load all
> > resources from the classpath, and it works ok inside the IDE. But when I
> > deploy the app I get the exception:
> >
> >
> > java.io.IOException: Resource not found:
> >
> > at
> >
> org.apache.lucene.analysis.util.ClasspathResourceLoader.openResource(Clas
> > spathResourceLoader.java:67)
> > at
> >
> org.apache.lucene.analysis.util.AbstractAnalysisFactory.getLines(AbstractAna
> > lysisFactory.java:254)
> > at
> >
> org.apache.lucene.analysis.util.AbstractAnalysisFactory.getWordSet(Abstract
> > AnalysisFactory.java:243)
> > at
> >
> org.apache.lucene.analysis.core.StopFilterFactory.inform(StopFilterFactory.ja
> > va:96)
> > at
> >
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.applyResourceL
> > oader(CustomAnalyzer.java:413)
> > at
> >
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.addTokenFilter(
> > CustomAnalyzer.java:308)
> > at
> >
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.addTokenFilter(
> > CustomAnalyzer.java:299)
> > at mysearchapp.Page.<init>(Page.java:36)
> > at mysearchapp.MySearchApp.main(MySearchApp.java:53)
> >
> >
> > When I check the classpath for the resources inside MANIFEST.MF
> everything
> > seems to be in order.
> >
> >
> >
> > Leonardo.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: CustomAnalyzer.Builder builder()

Uwe Schindler
Empty is not null:

>   .addTokenFilter(StopFilterFactory.class,
>     "ignoreCase", "true",
>     "words", "",
>     "format", "wordset")

This will cause the empty named file be loaded, which may not work with all class loaders. Just remove the useless parameters. Remove "words" and "format". In that case you can also go with the default builder() and you have no classloader issues.

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Leonardo Pérez Pulido [mailto:[hidden email]]
> Sent: Thursday, March 30, 2017 6:01 PM
> To: [hidden email]
> Subject: Re: CustomAnalyzer.Builder builder()
>
> Hi,
>
>
> I do not know to what file name do you refer, can you please be a little more
> specific? Did you mean a stopwords file name?
>
>
> Currently I am developing  an application which do not have a name yet, this
> is why you see the class MySearchApp, which is a test source file on which I
> am testing Lucene features, in this case the CustomAnalyzer.Builder class to
> build a custom analyzer.
>
>
> If you refer to the empty argument on StopFilterFactory.class, which we can
> use to pass a stopwords file to parse, the documentation says that all
> arguments are optional and in case of an empty stopwords file name, then
> Lucene will use StopAnalyzer.ENGLISH_STOP_WORDS_SET by default. Is is
> true?
>
>
> There is no secrets here, I am only trying to learn Lucene.
>
>
> Leonardo.
>
> ________________________________
> From: Uwe Schindler <[hidden email]>
> Sent: 30 March 2017 08:48:13
> To: [hidden email]
> Subject: RE: CustomAnalyzer.Builder builder()
>
> Hi,
>
> I am still a bit confused why you use an empty file name! Is this just
> copypasted here for privacy reasons wthout filename, or is there really no
> file name? This would explain why it may not work with the defaults.
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> http://www.thetaphi.de
> eMail: [hidden email]
>
> > -----Original Message-----
> > From: Leonardo Pérez Pulido [mailto:[hidden email]]
> > Sent: Thursday, March 30, 2017 2:30 PM
> > To: [hidden email]
> > Subject: Re: CustomAnalyzer.Builder builder()
> >
> > Hi,
> >
> >
> > Yes you are right, using the ClasspathResourceLoader class did solved the
> > issue, passing my on class as parameter:
> >
> >
> > ClasspathResourceLoader resourceLoader = new
> >           ClasspathResourceLoader(MySearchApp.class);
> > Analyzer analyzer = CustomAnalyzer.builder(resourceLoader)
> >   .withTokenizer(StandardTokenizerFactory.class)
> >   .addTokenFilter(StandardFilterFactory.class)
> >   .addTokenFilter(LowerCaseFilterFactory.class)
> >   .addTokenFilter(ASCIIFoldingFilterFactory.class)
> >   .addTokenFilter(StopFilterFactory.class,
> >     "ignoreCase", "true",
> >     "words", "",
> >     "format", "wordset")
> >   .build();
> >
> >
> > As you can see, what I am trying to achieve is to add
> ASCIIFolfingFilterFactory
> > to a custom analyzer chain.
> >
> > Thanks very much!
> >
> > Leonardo.
> >
> > ________________________________
> > From: Uwe Schindler <[hidden email]>
> > Sent: 30 March 2017 02:58 AM
> > To: [hidden email]
> > Subject: RE: CustomAnalyzer.Builder builder()
> >
> > Hi,
> >
> > the resources must be setup inside the JAR file in the same way like in
> > classpath of your IDE. As you did not give any examples and also removed
> > the resource name from your stack trace so we cannot figure out what you
> > are doing!
> >
> > The default builder uses the context classloader of current thread. This
> > means that your application server (runtime environment) has to be setup
> > correctly. In most cases for production use this is not ideal, as you have no
> > control where the classes are loaded from and leave it to the externally
> > defined properties. To fully define where the reosurces should be loaded
> > from, construct your own ClasspathResourceLoader, e.g. by giving your
> own
> > class as parameter. Then all resources are loaded directly from you Class's
> > package inside the JAR file.
> >
> > You say that you have the resources below META-INF: If you do this, the
> path
> > you pass to load the resources must also start with /META-INF! Resources
> in
> > Java have nothing to do with the META-INF folder. They are loaded like
> > classes from the root folder of the JAR, so the path must be relative to JAR
> > file's root.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> > http://www.thetaphi.de
> > eMail: [hidden email]
> >
> > > -----Original Message-----
> > > From: Leonardo Pérez Pulido [mailto:[hidden email]]
> > > Sent: Thursday, March 30, 2017 12:34 AM
> > > To: [hidden email]
> > > Subject: CustomAnalyzer.Builder builder()
> > >
> > > Hi,
> > >
> > >
> > > The documentation for the CustomAnalyzer Analyzer is very specific, one
> > can
> > > create an instance of a custom analyzer in three different ways.
> > >
> > >
> > > Using the default CustomAnalizer.builder(...), is supposed to load all
> > > resources from the classpath, and it works ok inside the IDE. But when I
> > > deploy the app I get the exception:
> > >
> > >
> > > java.io.IOException: Resource not found:
> > >
> > > at
> > >
> >
> org.apache.lucene.analysis.util.ClasspathResourceLoader.openResource(Clas
> > > spathResourceLoader.java:67)
> > > at
> > >
> >
> org.apache.lucene.analysis.util.AbstractAnalysisFactory.getLines(AbstractAna
> > > lysisFactory.java:254)
> > > at
> > >
> >
> org.apache.lucene.analysis.util.AbstractAnalysisFactory.getWordSet(Abstract
> > > AnalysisFactory.java:243)
> > > at
> > >
> >
> org.apache.lucene.analysis.core.StopFilterFactory.inform(StopFilterFactory.ja
> > > va:96)
> > > at
> > >
> >
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.applyResourceL
> > > oader(CustomAnalyzer.java:413)
> > > at
> > >
> >
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.addTokenFilter(
> > > CustomAnalyzer.java:308)
> > > at
> > >
> >
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.addTokenFilter(
> > > CustomAnalyzer.java:299)
> > > at mysearchapp.Page.<init>(Page.java:36)
> > > at mysearchapp.MySearchApp.main(MySearchApp.java:53)
> > >
> > >
> > > When I check the classpath for the resources inside MANIFEST.MF
> > everything
> > > seems to be in order.
> > >
> > >
> > >
> > > Leonardo.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: CustomAnalyzer.Builder builder()

leoperezpulido
Thank you Uwe, that was really helpful.


Leonardo.

________________________________
From: Uwe Schindler <[hidden email]>
Sent: 30 March 2017 13:14:17
To: [hidden email]
Subject: RE: CustomAnalyzer.Builder builder()

Empty is not null:

>   .addTokenFilter(StopFilterFactory.class,
>     "ignoreCase", "true",
>     "words", "",
>     "format", "wordset")

This will cause the empty named file be loaded, which may not work with all class loaders. Just remove the useless parameters. Remove "words" and "format". In that case you can also go with the default builder() and you have no classloader issues.

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Leonardo Pérez Pulido [mailto:[hidden email]]
> Sent: Thursday, March 30, 2017 6:01 PM
> To: [hidden email]
> Subject: Re: CustomAnalyzer.Builder builder()
>
> Hi,
>
>
> I do not know to what file name do you refer, can you please be a little more
> specific? Did you mean a stopwords file name?
>
>
> Currently I am developing  an application which do not have a name yet, this
> is why you see the class MySearchApp, which is a test source file on which I
> am testing Lucene features, in this case the CustomAnalyzer.Builder class to
> build a custom analyzer.
>
>
> If you refer to the empty argument on StopFilterFactory.class, which we can
> use to pass a stopwords file to parse, the documentation says that all
> arguments are optional and in case of an empty stopwords file name, then
> Lucene will use StopAnalyzer.ENGLISH_STOP_WORDS_SET by default. Is is
> true?
>
>
> There is no secrets here, I am only trying to learn Lucene.
>
>
> Leonardo.
>
> ________________________________
> From: Uwe Schindler <[hidden email]>
> Sent: 30 March 2017 08:48:13
> To: [hidden email]
> Subject: RE: CustomAnalyzer.Builder builder()
>
> Hi,
>
> I am still a bit confused why you use an empty file name! Is this just
> copypasted here for privacy reasons wthout filename, or is there really no
> file name? This would explain why it may not work with the defaults.
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> http://www.thetaphi.de
> eMail: [hidden email]
>
> > -----Original Message-----
> > From: Leonardo Pérez Pulido [mailto:[hidden email]]
> > Sent: Thursday, March 30, 2017 2:30 PM
> > To: [hidden email]
> > Subject: Re: CustomAnalyzer.Builder builder()
> >
> > Hi,
> >
> >
> > Yes you are right, using the ClasspathResourceLoader class did solved the
> > issue, passing my on class as parameter:
> >
> >
> > ClasspathResourceLoader resourceLoader = new
> >           ClasspathResourceLoader(MySearchApp.class);
> > Analyzer analyzer = CustomAnalyzer.builder(resourceLoader)
> >   .withTokenizer(StandardTokenizerFactory.class)
> >   .addTokenFilter(StandardFilterFactory.class)
> >   .addTokenFilter(LowerCaseFilterFactory.class)
> >   .addTokenFilter(ASCIIFoldingFilterFactory.class)
> >   .addTokenFilter(StopFilterFactory.class,
> >     "ignoreCase", "true",
> >     "words", "",
> >     "format", "wordset")
> >   .build();
> >
> >
> > As you can see, what I am trying to achieve is to add
> ASCIIFolfingFilterFactory
> > to a custom analyzer chain.
> >
> > Thanks very much!
> >
> > Leonardo.
> >
> > ________________________________
> > From: Uwe Schindler <[hidden email]>
> > Sent: 30 March 2017 02:58 AM
> > To: [hidden email]
> > Subject: RE: CustomAnalyzer.Builder builder()
> >
> > Hi,
> >
> > the resources must be setup inside the JAR file in the same way like in
> > classpath of your IDE. As you did not give any examples and also removed
> > the resource name from your stack trace so we cannot figure out what you
> > are doing!
> >
> > The default builder uses the context classloader of current thread. This
> > means that your application server (runtime environment) has to be setup
> > correctly. In most cases for production use this is not ideal, as you have no
> > control where the classes are loaded from and leave it to the externally
> > defined properties. To fully define where the reosurces should be loaded
> > from, construct your own ClasspathResourceLoader, e.g. by giving your
> own
> > class as parameter. Then all resources are loaded directly from you Class's
> > package inside the JAR file.
> >
> > You say that you have the resources below META-INF: If you do this, the
> path
> > you pass to load the resources must also start with /META-INF! Resources
> in
> > Java have nothing to do with the META-INF folder. They are loaded like
> > classes from the root folder of the JAR, so the path must be relative to JAR
> > file's root.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> > http://www.thetaphi.de
> > eMail: [hidden email]
> >
> > > -----Original Message-----
> > > From: Leonardo Pérez Pulido [mailto:[hidden email]]
> > > Sent: Thursday, March 30, 2017 12:34 AM
> > > To: [hidden email]
> > > Subject: CustomAnalyzer.Builder builder()
> > >
> > > Hi,
> > >
> > >
> > > The documentation for the CustomAnalyzer Analyzer is very specific, one
> > can
> > > create an instance of a custom analyzer in three different ways.
> > >
> > >
> > > Using the default CustomAnalizer.builder(...), is supposed to load all
> > > resources from the classpath, and it works ok inside the IDE. But when I
> > > deploy the app I get the exception:
> > >
> > >
> > > java.io.IOException: Resource not found:
> > >
> > > at
> > >
> >
> org.apache.lucene.analysis.util.ClasspathResourceLoader.openResource(Clas
> > > spathResourceLoader.java:67)
> > > at
> > >
> >
> org.apache.lucene.analysis.util.AbstractAnalysisFactory.getLines(AbstractAna
> > > lysisFactory.java:254)
> > > at
> > >
> >
> org.apache.lucene.analysis.util.AbstractAnalysisFactory.getWordSet(Abstract
> > > AnalysisFactory.java:243)
> > > at
> > >
> >
> org.apache.lucene.analysis.core.StopFilterFactory.inform(StopFilterFactory.ja
> > > va:96)
> > > at
> > >
> >
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.applyResourceL
> > > oader(CustomAnalyzer.java:413)
> > > at
> > >
> >
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.addTokenFilter(
> > > CustomAnalyzer.java:308)
> > > at
> > >
> >
> org.apache.lucene.analysis.custom.CustomAnalyzer$Builder.addTokenFilter(
> > > CustomAnalyzer.java:299)
> > > at mysearchapp.Page.<init>(Page.java:36)
> > > at mysearchapp.MySearchApp.main(MySearchApp.java:53)
> > >
> > >
> > > When I check the classpath for the resources inside MANIFEST.MF
> > everything
> > > seems to be in order.
> > >
> > >
> > >
> > > Leonardo.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Loading...