Using Solr w/Resin

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Using Solr w/Resin

kkrugler
Hi all, and especially Yonik :),

Just for grins, I'm trying Solr with Resin. I did the following:

1. Copied solr-nightly.war into <resin root>/webapps/.

2. Cloned/edited the resin.conf file to include a mapping for Solr, by adding:

       <web-app id='/solr/'>
        <document-directory>webapps/solr-nightly</document-directory>
       </web-app>

3. Started up Resin.

It created the <resin root>/webapps/solr-nightly directory, as
expected, but then when I tried to use the Solr servlet it complained
about not finding solrconfig.xml (see stack trace at end of email).

4. I realized that there was no solrconf folder included by default
in the .war, so I added it inside of the <resin
root>/webapps/solr-nightly directory. Same result. Tried moving it to
inside of WEB-INF, same thing.

5. I uncommented out the lines in the web.xml file that Yonik
indicated were necessary to get Resin to work properly with Solr, but
that didn't change anything.

So I figured I'd ask here about the "correct" way to configure Solr
with Resin, rather than continue to thrash. It would help if I was
more of a Resin jock, but so far my usage has been limited to futzing
with the Resin config file to add simple servlets.

Thanks,

-- Ken

PS - I'm using Resin-3.0.17, running on Mac OS X 10.4.5, if that matters.

====================================================================
java.lang.RuntimeException: Can't find resource solrconfig.xml
        at org.apache.solr.core.Config.openResource(Config.java:259)
        at org.apache.solr.core.SolrConfig.<clinit>(SolrConfig.java:33)
        at
org.apache.solr.update.SolrIndexConfig.<clinit>(SolrIndexConfig.java:34)
        at org.apache.solr.core.SolrCore.<clinit>(SolrCore.java:71)
        at org.apache.solr.servlet.SolrServlet.init(SolrServlet.java:58)
        at javax.servlet.GenericServlet.init(GenericServlet.java:69)
        at
com.caucho.server.dispatch.ServletConfigImpl.createServlet(ServletConfigImpl.java:600)
        at
com.caucho.server.dispatch.ServletManager.init(ServletManager.java:152)
        at com.caucho.server.webapp.Application.start(Application.java:1599)
        at
com.caucho.server.deploy.DeployController.startImpl(DeployController.java:579)
        at
com.caucho.server.deploy.DeployController.restartImpl(DeployController.java:542)
        at
com.caucho.server.deploy.StartAutoRedeployAutoStrategy.request(StartAutoRedeployAutoStrategy.java:125)
        at
com.caucho.server.deploy.DeployController.request(DeployController.java:512)
        at
com.caucho.server.webapp.ApplicationContainer.getApplication(ApplicationContainer.java:847)
        at
com.caucho.server.webapp.ApplicationContainer.buildInvocation(ApplicationContainer.java:687)
        at com.caucho.server.host.Host.buildInvocation(Host.java:421)
        at
com.caucho.server.host.HostContainer.buildInvocation(HostContainer.java:316)
        at
com.caucho.server.resin.ServletServer.buildInvocation(ServletServer.java:582)
        at
com.caucho.server.dispatch.DispatchServer.buildInvocation(DispatchServer.java:198)
        at
com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:262)
        at com.caucho.server.port.TcpConnection.run(TcpConnection.java:388)
        at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
        at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
        at java.lang.Thread.run(Thread.java:613)


--
Ken Krugler
Krugle, Inc.
+1 530-210-6378
"Find Code, Find Answers"
Reply | Threaded
Open this post in threaded view
|

Re: Using Solr w/Resin

Yonik Seeley
On 4/18/06, Ken Krugler <[hidden email]> wrote:
> 2. Cloned/edited the resin.conf file to include a mapping for Solr, by adding:
>
>        <web-app id='/solr/'>
>         <document-directory>webapps/solr-nightly</document-directory>
>        </web-app>

I normally just copy the war to solr.war, so I don't need the mapping...

> 3. Started up Resin.
>
> It created the <resin root>/webapps/solr-nightly directory, as
> expected, but then when I tried to use the Solr servlet it complained
> about not finding solrconfig.xml (see stack trace at end of email).

By default, Solr looks in the current-working-directory/solr/conf for
solrconfig.xml and schema.xml, and creates the index under
current-working-directory/solr/data

One way you can direct Solr to a different home location is setting
the solr.solr.home property...
./startup -Dsolr.solr.home=/foo/bar/solr

> 4. I realized that there was no solrconf folder included by default
> in the .war, so I added it inside of the <resin
> root>/webapps/solr-nightly directory. Same result. Tried moving it to
> inside of WEB-INF, same thing.
>
> 5. I uncommented out the lines in the web.xml file that Yonik
> indicated were necessary to get Resin to work properly with Solr, but
> that didn't change anything.

That part will come into play parsing solrconfig.xml and schema.xml

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: Using Solr w/Resin

Bill Au
Yonik is right.  You wouldn't need the web-app configuration in resin.conf
if you rename solr-nightly.war to solr.war.  Resin will expand all war files
in its webapps directory automatically.

For Resin, the default location for
conf files: <resin root>/solr/conf
index directory: <resin root>/solr/data

As Yonik has pointed out, you can override the default by adding
-Dsolr.solr.home=<your solr home> when starting resin/solr.  If
you use httpd.sh, it will look something like:

httpd.sh start -Dsolr.solr.home=/foo/bar/solr

By the way, the problem in Resin's XML parsed has been fixed in the
snapshot release and will be included in 3.0.19.

Bill

On 4/18/06, Yonik Seeley <[hidden email]> wrote:

>
> On 4/18/06, Ken Krugler <[hidden email]> wrote:
> > 2. Cloned/edited the resin.conf file to include a mapping for Solr, by
> adding:
> >
> >        <web-app id='/solr/'>
> >         <document-directory>webapps/solr-nightly</document-directory>
> >        </web-app>
>
> I normally just copy the war to solr.war, so I don't need the mapping...
>
> > 3. Started up Resin.
> >
> > It created the <resin root>/webapps/solr-nightly directory, as
> > expected, but then when I tried to use the Solr servlet it complained
> > about not finding solrconfig.xml (see stack trace at end of email).
>
> By default, Solr looks in the current-working-directory/solr/conf for
> solrconfig.xml and schema.xml, and creates the index under
> current-working-directory/solr/data
>
> One way you can direct Solr to a different home location is setting
> the solr.solr.home property...
> ./startup -Dsolr.solr.home=/foo/bar/solr
>
> > 4. I realized that there was no solrconf folder included by default
> > in the .war, so I added it inside of the <resin
> > root>/webapps/solr-nightly directory. Same result. Tried moving it to
> > inside of WEB-INF, same thing.
> >
> > 5. I uncommented out the lines in the web.xml file that Yonik
> > indicated were necessary to get Resin to work properly with Solr, but
> > that didn't change anything.
>
> That part will come into play parsing solrconfig.xml and schema.xml
>
> -Yonik
>
Reply | Threaded
Open this post in threaded view
|

Re: Using Solr w/Resin

kkrugler
In reply to this post by Yonik Seeley
>On 4/18/06, Ken Krugler <[hidden email]> wrote:
>>  2. Cloned/edited the resin.conf file to include a mapping for
>>Solr, by adding:
>>
>>         <web-app id='/solr/'>
>>          <document-directory>webapps/solr-nightly</document-directory>
>>         </web-app>
>
>I normally just copy the war to solr.war, so I don't need the mapping...
>
>>  3. Started up Resin.
>>
>>  It created the <resin root>/webapps/solr-nightly directory, as
>>  expected, but then when I tried to use the Solr servlet it complained
>>  about not finding solrconfig.xml (see stack trace at end of email).
>
>By default, Solr looks in the current-working-directory/solr/conf for
>solrconfig.xml and schema.xml,

 From the code, it appears to try:

  * $CWD/solrconf/
  * $CWD/
  * $CLASSPATH

Maybe my code is out of date, but I didn't see /solr/conf being used
as the path.

>and creates the index under
>current-working-directory/solr/data

 From the code, I see it being created at $CWD/data, where "data" is
the default if not specified in the solrconfig.xml.

>One way you can direct Solr to a different home location is setting
>the solr.solr.home property...
>./startup -Dsolr.solr.home=/foo/bar/solr

This didn't work for me. I tried a number of different permutations,
but maybe I missed the magic combination. When I grepped the source,
I didn't see this pattern (solr.solr.home) being used anywhere.

I was able to control the location of the config file by using
-Dsolr.configDir=<resin root>/webapps/solr/conf/ to set the location.

But the /data directory still gets created in <resin root>. I saw in
SolrServlet.java's init() method these lines:

     String configDir=getServletContext().getInitParameter("solr.configDir");
     String dataDir=getServletContext().getInitParameter("solr.dataDir");

Though the servlet doesn't seem to use either, and I didn't see
solr.dataDir being used in the code anywhere, and trying to control
it from the command line didn't seem to work.

I could uncomment & edit the <dataDir> tag in the solrconfig.xml
file, and that worked.

All of the above makes me worried that I'm missing something, or have
the wrong sources, but from what I can tell:

1. The only ways to control the location of the data directory (used
for the index) are...

a. Change the CWD - not sure how to do that for a webapp inside of Resin.

b. Edit the <dataDir> tag in the solrconfig.xml file. This also lets
you change the name of the directory.

2. The only ways to control the location of the config directory are...

a. Change the CWD.

b. Pass -Dsolr.configDir=xxx in the command line to Resin's httpd.sh
script. This also lets you change the name of the directory.

Thanks,

-- Ken
--
Ken Krugler
Krugle, Inc.
+1 530-210-6378
"Find Code, Find Answers"
Reply | Threaded
Open this post in threaded view
|

Re: Using Solr w/Resin

Chris Hostetter-3


: Maybe my code is out of date, but I didn't see /solr/conf being used
: as the path.

yeah .. this was changed starting with the 2006-04-01 nightly builds,
Yonik sent out an announcement about it on Mar31.  That's when the "solr
home" concept was introduced, and the data directory was changed to be
relative it.

i would suggest you upgrade to a new nightly build, and try using
-Dsolr.solr.home

: a. Change the CWD - not sure how to do that for a webapp inside of Resin.

FYI: there's really no way to do it per webapp, it's just a question of
what directory you start resin in -- that's one of the reasons the
solr.solr.home system property was created ... but we're realizing now
that we can't quite do what we thought we could to support multiple
instances in the same server instance, so an additional mechanism is
likely to be added soon -- more details can be found on the solr-dev list.

(The Solr code base is fairly stable ... but the packaging/configuration
of that code is still somewhat in flux, hence the lack of an official
release.  Prior to it being open sourced, the packaging was very CNET and
RPM specific.)




-Hoss

Reply | Threaded
Open this post in threaded view
|

Re: Using Solr w/Resin

kkrugler
>: Maybe my code is out of date, but I didn't see /solr/conf being used
>: as the path.
>
>yeah .. this was changed starting with the 2006-04-01 nightly builds,
>Yonik sent out an announcement about it on Mar31.  That's when the "solr
>home" concept was introduced, and the data directory was changed to be
>relative it.
>
>i would suggest you upgrade to a new nightly build, and try using
>-Dsolr.solr.home
>
>: a. Change the CWD - not sure how to do that for a webapp inside of Resin.
>
>FYI: there's really no way to do it per webapp, it's just a question of
>what directory you start resin in -- that's one of the reasons the
>solr.solr.home system property was created ... but we're realizing now
>that we can't quite do what we thought we could to support multiple
>instances in the same server instance, so an additional mechanism is
>likely to be added soon -- more details can be found on the solr-dev list.

I think I just ran into this...

I'd like to run two copies of Solr from inside Resin, since I have
two sets of searchable data with very distinct schemas/usage
patterns. But I can only specify one location for the configuration
directory, at least using either solr.configdir or solr.solr.home.

Once I'm subscribed to the dev list, I'll post my question about
timing on a fix versus doing something quick now for my needs. FWIW,
the quick fix for me is to pass in a directory location as an init()
parameter (via the servlet's web.xml)

Thanks,

-- Ken
--
Ken Krugler
Krugle, Inc.
+1 530-210-6378
"Find Code, Find Answers"