Can I use configsets with custom stopwords per collection?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Can I use configsets with custom stopwords per collection?

O. Klein
Im using Solr standalone and I want to use shared stopwords and custom
stopwords per collection. Is this possible?



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Can I use configsets with custom stopwords per collection?

Alexandre Rafalovitch
The stopwords are defined at the field type level as part of the
analyzer chain. So, you have per-field granularity. Not just
per-collection.

As stop-words are using files (though we have managed version as well,
you can share or not-share as much as you want even across different
field type definitions.

https://lucene.apache.org/solr/guide/7_5/filter-descriptions.html#stop-filter

Regards,
   Alex.
On Mon, 3 Dec 2018 at 10:04, O. Klein <[hidden email]> wrote:
>
> Im using Solr standalone and I want to use shared stopwords and custom
> stopwords per collection. Is this possible?
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Can I use configsets with custom stopwords per collection?

O. Klein
Yeah, but if i define them in the schema of configset, The custom file with
stopwords is in a directory relative to the collection and not in configset.

So is there a way to define a path to stopwords with the collection as a
variable?



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Can I use configsets with custom stopwords per collection?

Alexandre Rafalovitch
I am not sure I fully understand what you are saying.

When you create a collection based on a configset, all the files
should be copied, including the stopwords.

You can also provide an absolute path.

Solr also supports variable substitutions (as seen in solrconfig.xml
library statements), but I am not actually sure if that applies to
managed-schema. Could be an interesting test.

Finally, you could use Managed configuration and push whatever set you
need to whatever collection, but that's an extra step to be managed
externally.

Regards,
   Alex.
On Mon, 3 Dec 2018 at 10:51, O. Klein <[hidden email]> wrote:

>
> Yeah, but if i define them in the schema of configset, The custom file with
> stopwords is in a directory relative to the collection and not in configset.
>
> So is there a way to define a path to stopwords with the collection as a
> variable?
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Can I use configsets with custom stopwords per collection?

Alexandre Rafalovitch
Actually, just to correct myself. Solr uses configset in two different
ways (very unfortunate):
1) When you do bin/solr create -c name -d configset, in which case the
content of configset directory is copied
2) When you actually link to a configset as a common configuration, in
which case I think nothing is copied and core.properties points back
to the shared configset. In this case, any modifications affect all
the cores using the same backing configset.

I am guessing in the second interpretation, your question makes more
sense. I would then look at substitution variables and or managed stop
words, but I do not know the expected behavior.

Regards,
   Alex.
On Mon, 3 Dec 2018 at 11:05, Alexandre Rafalovitch <[hidden email]> wrote:

>
> I am not sure I fully understand what you are saying.
>
> When you create a collection based on a configset, all the files
> should be copied, including the stopwords.
>
> You can also provide an absolute path.
>
> Solr also supports variable substitutions (as seen in solrconfig.xml
> library statements), but I am not actually sure if that applies to
> managed-schema. Could be an interesting test.
>
> Finally, you could use Managed configuration and push whatever set you
> need to whatever collection, but that's an extra step to be managed
> externally.
>
> Regards,
>    Alex.
> On Mon, 3 Dec 2018 at 10:51, O. Klein <[hidden email]> wrote:
> >
> > Yeah, but if i define them in the schema of configset, The custom file with
> > stopwords is in a directory relative to the collection and not in configset.
> >
> > So is there a way to define a path to stopwords with the collection as a
> > variable?
> >
> >
> >
> > --
> > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Can I use configsets with custom stopwords per collection?

O. Klein
Yeah, I'm not copying files. I want all collections to use 1 schema.

So I wonder, do managed stopwords work with configsets and store stopwords
per collection?

Also, what would be the substitution variable for collection name? Is there
a list somewhere?

Thanks!



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Can I use configsets with custom stopwords per collection?

Erick Erickson
Substitution variables are whatever you want. The file looks like:
${my.var.here:default_if_not_spcified}

then set it when you start Solr by
java .....   -Dmy.var.here=whatever  ......

Best,
Erick
On Tue, Dec 4, 2018 at 2:43 AM O. Klein <[hidden email]> wrote:

>
> Yeah, I'm not copying files. I want all collections to use 1 schema.
>
> So I wonder, do managed stopwords work with configsets and store stopwords
> per collection?
>
> Also, what would be the substitution variable for collection name? Is there
> a list somewhere?
>
> Thanks!
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Can I use configsets with custom stopwords per collection?

O. Klein
Ok. So with these suggestions, I found
https://lucene.apache.org/solr/guide/6_6/configuring-solrconfig-xml.html#Configuringsolrconfig.xml-ImplicitCoreProperties
So to test this I tried to use it in DIH as this has a similar issue with
configsets as every collection needs its own DIH.properties.

<propertyWriter dateFormat="dd-MM-yyyy HH:mm:ss"
type="SimplePropertiesWriter"            
directory="/var/solr/data/${solr.core.name}/conf" filename="DIH.properties"
locale="nl-NL" />

However does not work. Substituting ${solr.core.name} with core name, does
work.

Am I missing something?



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html