Separate config and index per webapp

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Separate config and index per webapp

Michael Levy-3
I'd like to use Solr for a number of separate projects running on a
single Tomcat instance; I thought I would have Solr running a separate
webapp for each project.

I understand that the default location for the solrconfig.xml and
schema.xml files and for the index are under Tomcat's
current-working-directory/solr/conf

I am hoping I can change the default location for each webapp.  Thanks!
Reply | Threaded
Open this post in threaded view
|

Re: Separate config and index per webapp

Yonik Seeley
On 5/15/06, Michael Levy <[hidden email]> wrote:
> I'd like to use Solr for a number of separate projects running on a
> single Tomcat instance; I thought I would have Solr running a separate
> webapp for each project.
>
> I understand that the default location for the solrconfig.xml and
> schema.xml files and for the index are under Tomcat's
> current-working-directory/solr/conf
>
> I am hoping I can change the default location for each webapp.  Thanks!

It's not yet possible, but see this thread:
http://www.mail-archive.com/solr-dev@.../msg00298.html

It looks like JNDI offers the easiest portable way.
The main SolrServlet would then need to be modified to do a JNDI
lookup and base config off of that.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: Separate config and index per webapp

Mike Baranczak-2
On May 15, 2006, at 3:26 PM, Yonik Seeley wrote:

> On 5/15/06, Michael Levy <[hidden email]> wrote:
>> I'd like to use Solr for a number of separate projects running on a
>> single Tomcat instance; I thought I would have Solr running a  
>> separate
>> webapp for each project.
>>
>> I understand that the default location for the solrconfig.xml and
>> schema.xml files and for the index are under Tomcat's
>> current-working-directory/solr/conf
>>
>> I am hoping I can change the default location for each webapp.  
>> Thanks!
>
> It's not yet possible, but see this thread:
> http://www.mail-archive.com/solr-dev@.../msg00298.html
>
> It looks like JNDI offers the easiest portable way.
> The main SolrServlet would then need to be modified to do a JNDI
> lookup and base config off of that.


The way I did it is I unpacked the WAR and put my config files into  
WEB-INF/classes/. Then I individually edit each app's solrconfig.xml  
and specify a unique path for the index directory. This may not be  
for everyone, but since I had to unpack the WAR anyway...

-MB

Reply | Threaded
Open this post in threaded view
|

Re: Separate config and index per webapp

kkrugler
In reply to this post by Yonik Seeley
>On 5/15/06, Michael Levy <[hidden email]> wrote:
>>I'd like to use Solr for a number of separate projects running on a
>>single Tomcat instance; I thought I would have Solr running a separate
>>webapp for each project.
>>
>>I understand that the default location for the solrconfig.xml and
>>schema.xml files and for the index are under Tomcat's
>>current-working-directory/solr/conf
>>
>>I am hoping I can change the default location for each webapp.  Thanks!
>
>It's not yet possible, but see this thread:
>http://www.mail-archive.com/solr-dev@.../msg00298.html
>
>It looks like JNDI offers the easiest portable way.
>The main SolrServlet would then need to be modified to do a JNDI
>lookup and base config off of that.

I worked around it this way:

http://www.mail-archive.com/solr-dev@.../msg00321.html

-- Ken
--
Ken Krugler
Krugle, Inc.
+1 530-210-6378
"Find Code, Find Answers"
Reply | Threaded
Open this post in threaded view
|

Re: Separate config and index per webapp

Marcus Stratmann
In reply to this post by Yonik Seeley
Yonik Seeley wrote:
>> I am hoping I can change the default location for each webapp.  Thanks!
> It's not yet possible, but see this thread:
> http://www.mail-archive.com/solr-dev@.../msg00298.html
If I see it right, if I just rename the webapp to, say, "solrfoo" then
it still uses the system property solr.solr.home to search for the
configuration, *not* solrfoo.solr.home, right?
I'm searching for a way to have multiple webapps with different
configuration, too. I would really appreciate if that could be made
possible. (And sorry, I'd really like to do it myself, but my java
knowledge doesn not suffice for that.)

Another thing I would like to see is a complete detachment of the solr
configuration of that of the servlet container. Currently I have to
change the path to the configuration files by setting solr.solr.home or
(even worse!) by starting Tomcat (which I use) from it's base home dir.
A while ago I proposed to put solr's config into /etc/solr (for linux).
It was easily done (even for me) to add this directory to the places
being searched in Config.java. I thing if this is put in *additionally*
it should be no problem even for those people who just want to try out
solr and have no root privileges.

Marcus
Reply | Threaded
Open this post in threaded view
|

Re: Separate config and index per webapp

Chris Hostetter-3

: If I see it right, if I just rename the webapp to, say, "solrfoo" then
: it still uses the system property solr.solr.home to search for the
: configuration, *not* solrfoo.solr.home, right?

correct .. we thought we can impliment something that looked at the war
file name easily ... but then we were set straight -- there is no portable
way to do that, hence we came up with the current JNDI plan which isn't
quite as "out of the box" as we had hoped, but it has the advantage of
being possible.

: Another thing I would like to see is a complete detachment of the solr
: configuration of that of the servlet container. Currently I have to

I don't know that we'll ever be able to make configuring Solr completely
detatched from configuring the servlet container -- other then the
simplest method of putting your.  Personally i don't think that should be
a major goal: a well tuned Solr installation is going to require that you
consider/configure your servlet container's heap size to meet your needs
anyway.

: change the path to the configuration files by setting solr.solr.home or
: (even worse!) by starting Tomcat (which I use) from it's base home dir.

just to clarify: if you only want one instance of Solr on the port, you
do't *have* to start tomcat from it's base directory ... you just have to
make sure the "solr" directory is in whatever the current working
directory is when you do start it.  many servlet container start scripts
already make sure to set the working directory to the installation
directory before starting up the server, so maybe that's the source of the
confusion?

: A while ago I proposed to put solr's config into /etc/solr (for linux).
: It was easily done (even for me) to add this directory to the places
: being searched in Config.java. I thing if this is put in *additionally*
: it should be no problem even for those people who just want to try out
: solr and have no root privileges.

1) I'm not sure how this would work cleanly for people on Windows boxes
2) this still wouldn't make it possible to suport multiple webapps on a
single port (it would acctually make it hard to support multiple ports on
the same box).

If the JNDI approach gets implimented, then it should make it easy for you
to specify /etc/solr (or any other directory) as your config directory
with a one line change to your tomcat configuration.

But please don't let me discourage you from experiencing with this
approach yourself, and submitting a patch that does what you'd like ...
Perhaps I'm just missunderstanding.




-Hoss

Reply | Threaded
Open this post in threaded view
|

Re: Separate config and index per webapp

Michael Levy-3
In reply to this post by Mike Baranczak-2
I copied solrconfig.xml, schema.xml, synonyms.txt, protwords.txt and
stopwords.txt into the WEB-INF/classes directory and the webapp is
reading those files.

I was thinking that if I uncommented the indexDir element in
solrconfig.xml, it would then use that directory in which to place the index
The commented-out default value is:
  <indexDir>index</indexDir>

I tried changing the value to:
  <indexDir>indexXXX</indexDir>

However this is being ignored; the index directory is still being
created under Tomcat's current-working-directory/solr/conf/index   (In
my case I'm testing under Windows and it's C:\Tomcat 5.5\solr\data\index)

Any ideas?  Thanks.

Mike Baranczak wrote:

> On May 15, 2006, at 3:26 PM, Yonik Seeley wrote:
>
>> On 5/15/06, Michael Levy <[hidden email]> wrote:
>>> I'd like to use Solr for a number of separate projects running on a
>>> single Tomcat instance; I thought I would have Solr running a separate
>>> webapp for each project.
>>>
>>> I understand that the default location for the solrconfig.xml and
>>> schema.xml files and for the index are under Tomcat's
>>> current-working-directory/solr/conf
>>>
>>> I am hoping I can change the default location for each webapp.  Thanks!
>>
>> It's not yet possible, but see this thread:
>> http://www.mail-archive.com/solr-dev@.../msg00298.html
>>
>> It looks like JNDI offers the easiest portable way.
>> The main SolrServlet would then need to be modified to do a JNDI
>> lookup and base config off of that.
>
>
> The way I did it is I unpacked the WAR and put my config files into
> WEB-INF/classes/. Then I individually edit each app's solrconfig.xml
> and specify a unique path for the index directory. This may not be for
> everyone, but since I had to unpack the WAR anyway...
>
> -MB
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Separate config and index per webapp

Yonik Seeley
Try using the latest nightly build, and changing indexDir to dataDir:

  <!-- Used to specify an alternate directory to hold all index data
       other than the default ./solr/data
       If replication is in use, this should match the replication configuration
. -->
  <!--
  <dataDir>./solr/data</dataDir>
  -->

-Yonik

On 5/17/06, Michael Levy <[hidden email]> wrote:

> I copied solrconfig.xml, schema.xml, synonyms.txt, protwords.txt and
> stopwords.txt into the WEB-INF/classes directory and the webapp is
> reading those files.
>
> I was thinking that if I uncommented the indexDir element in
> solrconfig.xml, it would then use that directory in which to place the index
> The commented-out default value is:
>   <indexDir>index</indexDir>
>
> I tried changing the value to:
>   <indexDir>indexXXX</indexDir>
>
> However this is being ignored; the index directory is still being
> created under Tomcat's current-working-directory/solr/conf/index   (In
> my case I'm testing under Windows and it's C:\Tomcat 5.5\solr\data\index)
>
> Any ideas?  Thanks.
>
> Mike Baranczak wrote:
> > On May 15, 2006, at 3:26 PM, Yonik Seeley wrote:
> >
> >> On 5/15/06, Michael Levy <[hidden email]> wrote:
> >>> I'd like to use Solr for a number of separate projects running on a
> >>> single Tomcat instance; I thought I would have Solr running a separate
> >>> webapp for each project.
> >>>
> >>> I understand that the default location for the solrconfig.xml and
> >>> schema.xml files and for the index are under Tomcat's
> >>> current-working-directory/solr/conf
> >>>
> >>> I am hoping I can change the default location for each webapp.  Thanks!
> >>
> >> It's not yet possible, but see this thread:
> >> http://www.mail-archive.com/solr-dev@.../msg00298.html
> >>
> >> It looks like JNDI offers the easiest portable way.
> >> The main SolrServlet would then need to be modified to do a JNDI
> >> lookup and base config off of that.
> >
> >
> > The way I did it is I unpacked the WAR and put my config files into
> > WEB-INF/classes/. Then I individually edit each app's solrconfig.xml
> > and specify a unique path for the index directory. This may not be for
> > everyone, but since I had to unpack the WAR anyway...
> >
> > -MB
Reply | Threaded
Open this post in threaded view
|

Re: Separate config and index per webapp

Mike Baranczak-2
Yeah, the config that I have (based on the repo version from a couple  
weeks ago) has <dataDir> set, and it works as expected. I don't know  
where the indexDir came from.


On May 17, 2006, at 2:56 PM, Yonik Seeley wrote:

> Try using the latest nightly build, and changing indexDir to dataDir:
>
>  <!-- Used to specify an alternate directory to hold all index data
>       other than the default ./solr/data
>       If replication is in use, this should match the replication  
> configuration
> . -->
>  <!--
>  <dataDir>./solr/data</dataDir>
>  -->
>
> -Yonik
>
> On 5/17/06, Michael Levy <[hidden email]> wrote:
>> I copied solrconfig.xml, schema.xml, synonyms.txt, protwords.txt and
>> stopwords.txt into the WEB-INF/classes directory and the webapp is
>> reading those files.
>>
>> I was thinking that if I uncommented the indexDir element in
>> solrconfig.xml, it would then use that directory in which to place  
>> the index
>> The commented-out default value is:
>>   <indexDir>index</indexDir>
>>
>> I tried changing the value to:
>>   <indexDir>indexXXX</indexDir>
>>
>> However this is being ignored; the index directory is still being
>> created under Tomcat's current-working-directory/solr/conf/index    
>> (In
>> my case I'm testing under Windows and it's C:\Tomcat 5.5\solr\data
>> \index)
>>
>> Any ideas?  Thanks.
>>
>> Mike Baranczak wrote:
>> > On May 15, 2006, at 3:26 PM, Yonik Seeley wrote:
>> >
>> >> On 5/15/06, Michael Levy <[hidden email]> wrote:
>> >>> I'd like to use Solr for a number of separate projects running  
>> on a
>> >>> single Tomcat instance; I thought I would have Solr running a  
>> separate
>> >>> webapp for each project.
>> >>>
>> >>> I understand that the default location for the solrconfig.xml and
>> >>> schema.xml files and for the index are under Tomcat's
>> >>> current-working-directory/solr/conf
>> >>>
>> >>> I am hoping I can change the default location for each  
>> webapp.  Thanks!
>> >>
>> >> It's not yet possible, but see this thread:
>> >> http://www.mail-archive.com/solr-dev@.../ 
>> msg00298.html
>> >>
>> >> It looks like JNDI offers the easiest portable way.
>> >> The main SolrServlet would then need to be modified to do a JNDI
>> >> lookup and base config off of that.
>> >
>> >
>> > The way I did it is I unpacked the WAR and put my config files into
>> > WEB-INF/classes/. Then I individually edit each app's  
>> solrconfig.xml
>> > and specify a unique path for the index directory. This may not  
>> be for
>> > everyone, but since I had to unpack the WAR anyway...
>> >
>> > -MB

Reply | Threaded
Open this post in threaded view
|

Re: Separate config and index per webapp

Michael Levy-3
In reply to this post by Yonik Seeley
Changing to <dataDir> works fine with my version from 2006-05-15

Thanks for your help, this is just what I needed.

Yonik Seeley wrote:

> Try using the latest nightly build, and changing indexDir to dataDir:
>
>  <!-- Used to specify an alternate directory to hold all index data
>       other than the default ./solr/data
>       If replication is in use, this should match the replication
> configuration
> . -->
>  <!--
>  <dataDir>./solr/data</dataDir>
>  -->
>
> -Yonik
>
> On 5/17/06, Michael Levy <[hidden email]> wrote:
>> I copied solrconfig.xml, schema.xml, synonyms.txt, protwords.txt and
>> stopwords.txt into the WEB-INF/classes directory and the webapp is
>> reading those files.
>>
>> I was thinking that if I uncommented the indexDir element in
>> solrconfig.xml, it would then use that directory in which to place
>> the index
>> The commented-out default value is:
>>   <indexDir>index</indexDir>
>>
>> I tried changing the value to:
>>   <indexDir>indexXXX</indexDir>
>>
>> However this is being ignored; the index directory is still being
>> created under Tomcat's current-working-directory/solr/conf/index   (In
>> my case I'm testing under Windows and it's C:\Tomcat
>> 5.5\solr\data\index)
>>
>> Any ideas?  Thanks.
>>
>> Mike Baranczak wrote:
>> > On May 15, 2006, at 3:26 PM, Yonik Seeley wrote:
>> >
>> >> On 5/15/06, Michael Levy <[hidden email]> wrote:
>> >>> I'd like to use Solr for a number of separate projects running on a
>> >>> single Tomcat instance; I thought I would have Solr running a
>> separate
>> >>> webapp for each project.
>> >>>
>> >>> I understand that the default location for the solrconfig.xml and
>> >>> schema.xml files and for the index are under Tomcat's
>> >>> current-working-directory/solr/conf
>> >>>
>> >>> I am hoping I can change the default location for each webapp.  
>> Thanks!
>> >>
>> >> It's not yet possible, but see this thread:
>> >> http://www.mail-archive.com/solr-dev@.../msg00298.html
>> >>
>> >> It looks like JNDI offers the easiest portable way.
>> >> The main SolrServlet would then need to be modified to do a JNDI
>> >> lookup and base config off of that.
>> >
>> >
>> > The way I did it is I unpacked the WAR and put my config files into
>> > WEB-INF/classes/. Then I individually edit each app's solrconfig.xml
>> > and specify a unique path for the index directory. This may not be for
>> > everyone, but since I had to unpack the WAR anyway...
>> >
>> > -MB
>
Reply | Threaded
Open this post in threaded view
|

Re: Separate config and index per webapp

Marcus Stratmann
In reply to this post by Chris Hostetter-3
Chris Hostetter wrote:
> correct .. we thought we can impliment something that looked at the war
> file name easily ... but then we were set straight -- there is no portable
> way to do that, hence we came up with the current JNDI plan which isn't
> quite as "out of the box" as we had hoped, but it has the advantage of
> being possible.
Yes, I observed the discussion on the developer mailing list for a while
and was suprised to read that there isn't an easy solution for this problem.

> I don't know that we'll ever be able to make configuring Solr completely
> detatched from configuring the servlet container -- other then the
> simplest method of putting your.  Personally i don't think that should be
> a major goal: a well tuned Solr installation is going to require that you
> consider/configure your servlet container's heap size to meet your needs
> anyway.
Good point. Currently I'm using the solr.solr.home system property and
besides the heap size it is the only Solr specific configuration I have
to do with Tomcat. So I can live with that.

> just to clarify: if you only want one instance of Solr on the port, you
> do't *have* to start tomcat from it's base directory
I know, I just wanted to point out that somehow Tomcat is involved in
the Solr configuration.

> ... you just have to
> make sure the "solr" directory is in whatever the current working
> directory is when you do start it.
But what if another webapp needs the server to be started from
/some/directory/it/likes ?

> If the JNDI approach gets implimented, then it should make it easy for you
> to specify /etc/solr (or any other directory) as your config directory
> with a one line change to your tomcat configuration.
I'm looking forward to that. :-)

Thanks,
Marcus