multiple solr webapps

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

multiple solr webapps

Yonik Seeley
I've been looking into how to have multiple solr webapps, and have
them default to looking into different spots for config (and looking
at a different property to override that).

Unfortunately, it doesn't look like there is a way to do this from the
servlet API.
It looks like the only course of action remaining is to drive the
differentiation off of something visible to the webapp, like a context
param in it's web.xml, which means people will have to explode the
webapp and edit web.xml.

It's not ideal, but it's only for the minority who need multiple solr
webapps in the same JVM.

Thoughts?

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: multiple solr webapps

Chris Hostetter-3
: I've been looking into how to have multiple solr webapps, and have
: them default to looking into different spots for config (and looking
: at a different property to override that).
:
: Unfortunately, it doesn't look like there is a way to do this from the
: servlet API.

Can't we just use ServletContext.getServletContextName(), and then build a
relative path (or a system property name to look up a path) from that?

(or does ServletContext.getServletContextName() not do what I think i
remember it doing)

-Hoss

Reply | Threaded
Open this post in threaded view
|

Re: multiple solr webapps

Yonik Seeley
On 4/18/06, Chris Hostetter <[hidden email]> wrote:
> (or does ServletContext.getServletContextName() not do what I think i
> remember it doing)

Unfortunately not.... the javadoc says it comes from the web.xml

 java.lang.String getServletContextName()
          Returns the name of this web application corresponding to
this ServletContext as specified in the deployment descriptor for this
web application by the display-name element.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: multiple solr webapps

Chris Hostetter-3

: Unfortunately not.... the javadoc says it comes from the web.xml
:
:  java.lang.String getServletContextName()
:           Returns the name of this web application corresponding to
: this ServletContext as specified in the deployment descriptor for this
: web application by the display-name element.

Yeah, but if we don't put a display name in the web.xml, it's the name of
the webapp directory.  (or at least that's the way it works on Resin).

There's also HttpServletRequest.getContextPath() ..  it will work
regardless of display-name, but it requires a request object.

Toss the attached index.jsp into a couple of differnet webapp
directories, then toss the web.xml into one of those webapps, and you'll
see what i mean.

index.jsp (254 bytes) Download Attachment
web.xml (166 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: multiple solr webapps

Yoav Shapira-2
In reply to this post by Yonik Seeley
Hola,

No, don't use getServletContextName().  Besides being optional, it
doesn't necessarily related to a path on the disk.  For that matter,
don't use the disk path approach anyhow, as things get quirky /
fragile for users running packed WARs.  There are a couple of
alternatives, one using the classpath and one using the
ServletContext#getResource approach.

The classpath one is the standard Java classpath resource lookup
mechanism: in your class, do
getClass().getResource("path.to.your.resource") -- in a webpp, this
will automatically use the webapp's specific classloader by default,
so if you have a config file in WEB-INF/lib or WEB-INF/classes, it
will get picked up.  (And conveniently, one could specify server-wide
Solr defaults in a higher classloader like the common server one).

The ServletContext one is very similar: use ServletContext.getResource
with the same path semantics, and a resource anywhere under your
webapp root will be fetched.

To compare / contrast the two: the ServletContext approach depends on
a servlet container, i.e. is hard to use and test from a command-line
utility.  It also doesn't provide the hierarchical defaulting
mechanism as easily as the classpath one.  On the flip side, some
people only like to have classes on the classpath in the name of
"neatness" although in reality, there are a lot of things (for example
DTD, XSD spec files) on the classpath at runtime.

If you need a more concrete example of how to do this, I'll be glad to
provide one.

Yoav


On 4/18/06, Yonik Seeley <[hidden email]> wrote:

> On 4/18/06, Chris Hostetter <[hidden email]> wrote:
> > (or does ServletContext.getServletContextName() not do what I think i
> > remember it doing)
>
> Unfortunately not.... the javadoc says it comes from the web.xml
>
>  java.lang.String       getServletContextName()
>           Returns the name of this web application corresponding to
> this ServletContext as specified in the deployment descriptor for this
> web application by the display-name element.
>
> -Yonik
>


--
Yoav Shapira
Nimalex LLC
1 Mifflin Place, Suite 310
Cambridge, MA, USA
[hidden email] / www.yoavshapira.com
Reply | Threaded
Open this post in threaded view
|

Re: multiple solr webapps

Yonik Seeley
On 4/18/06, Yoav Shapira <[hidden email]> wrote:
> There are a couple of
> alternatives, one using the classpath and one using the
> ServletContext#getResource approach.

But the goal was to have the exact same solr WAR, just copied to
different names... so we need some way to find out the <name> in
<name>.war

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: multiple solr webapps

Yoav Shapira-2
Hola,

> But the goal was to have the exact same solr WAR, just copied to
> different names... so we need some way to find out the <name> in
> <name>.war

There's no portable way of doing that, quite on purpose: the webapp is
supposed to run the same independent of its server admin-dependent
deployment details like WAR (or no WAR at all in the case of a
filesystem or DB deployment) packaging.

Another approach to consider, then, is using the server's JNDI
resources.  The code in the WAR would be the same, looking up the
configuration file URL (or any other JNDI resource) using the
java:comp/env space (which is different for every webapp, and portable
across all J2EE servers/servlet containers), but the server's
configuration file would specify different values for different
webapps.  It's not zero configuration like just copying a WAR would
be, but fairly close.

Yoav
Reply | Threaded
Open this post in threaded view
|

Re: multiple solr webapps

Bill Au
In reply to this post by Yoav Shapira-2
But shouldn't the return value of HttpServletRequest.getContextPath()
be the same independing of packaging (WAR or no WAR)?

Bill

On 4/18/06, Yoav Shapira <[hidden email]> wrote:

>
> Hola,
>
> No, don't use getServletContextName().  Besides being optional, it
> doesn't necessarily related to a path on the disk.  For that matter,
> don't use the disk path approach anyhow, as things get quirky /
> fragile for users running packed WARs.  There are a couple of
> alternatives, one using the classpath and one using the
> ServletContext#getResource approach.
>
> The classpath one is the standard Java classpath resource lookup
> mechanism: in your class, do
> getClass().getResource("path.to.your.resource") -- in a webpp, this
> will automatically use the webapp's specific classloader by default,
> so if you have a config file in WEB-INF/lib or WEB-INF/classes, it
> will get picked up.  (And conveniently, one could specify server-wide
> Solr defaults in a higher classloader like the common server one).
>
> The ServletContext one is very similar: use ServletContext.getResource
> with the same path semantics, and a resource anywhere under your
> webapp root will be fetched.
>
> To compare / contrast the two: the ServletContext approach depends on
> a servlet container, i.e. is hard to use and test from a command-line
> utility.  It also doesn't provide the hierarchical defaulting
> mechanism as easily as the classpath one.  On the flip side, some
> people only like to have classes on the classpath in the name of
> "neatness" although in reality, there are a lot of things (for example
> DTD, XSD spec files) on the classpath at runtime.
>
> If you need a more concrete example of how to do this, I'll be glad to
> provide one.
>
> Yoav
>
>
> On 4/18/06, Yonik Seeley <[hidden email]> wrote:
> > On 4/18/06, Chris Hostetter <[hidden email]> wrote:
> > > (or does ServletContext.getServletContextName() not do what I think i
> > > remember it doing)
> >
> > Unfortunately not.... the javadoc says it comes from the web.xml
> >
> >  java.lang.String       getServletContextName()
> >           Returns the name of this web application corresponding to
> > this ServletContext as specified in the deployment descriptor for this
> > web application by the display-name element.
> >
> > -Yonik
> >
>
>
> --
> Yoav Shapira
> Nimalex LLC
> 1 Mifflin Place, Suite 310
> Cambridge, MA, USA
> [hidden email] / www.yoavshapira.com
>
Reply | Threaded
Open this post in threaded view
|

Re: multiple solr webapps

Yoav Shapira-2
Yeah, it should.  IMHO that call was only added as a kowtow to people
demanding an easy hack approach -- and there's something to be said
for that, if many people demand it.  But it was still added
half-heartedly, as it requires a request, i.e. you can't use it an
initialization time before any requests come have arrived...

Yoav

On 4/18/06, Bill Au <[hidden email]> wrote:

> But shouldn't the return value of HttpServletRequest.getContextPath()
> be the same independing of packaging (WAR or no WAR)?
>
> Bill
>
> On 4/18/06, Yoav Shapira <[hidden email]> wrote:
> >
> > Hola,
> >
> > No, don't use getServletContextName().  Besides being optional, it
> > doesn't necessarily related to a path on the disk.  For that matter,
> > don't use the disk path approach anyhow, as things get quirky /
> > fragile for users running packed WARs.  There are a couple of
> > alternatives, one using the classpath and one using the
> > ServletContext#getResource approach.
> >
> > The classpath one is the standard Java classpath resource lookup
> > mechanism: in your class, do
> > getClass().getResource("path.to.your.resource") -- in a webpp, this
> > will automatically use the webapp's specific classloader by default,
> > so if you have a config file in WEB-INF/lib or WEB-INF/classes, it
> > will get picked up.  (And conveniently, one could specify server-wide
> > Solr defaults in a higher classloader like the common server one).
> >
> > The ServletContext one is very similar: use ServletContext.getResource
> > with the same path semantics, and a resource anywhere under your
> > webapp root will be fetched.
> >
> > To compare / contrast the two: the ServletContext approach depends on
> > a servlet container, i.e. is hard to use and test from a command-line
> > utility.  It also doesn't provide the hierarchical defaulting
> > mechanism as easily as the classpath one.  On the flip side, some
> > people only like to have classes on the classpath in the name of
> > "neatness" although in reality, there are a lot of things (for example
> > DTD, XSD spec files) on the classpath at runtime.
> >
> > If you need a more concrete example of how to do this, I'll be glad to
> > provide one.
> >
> > Yoav
> >
> >
> > On 4/18/06, Yonik Seeley <[hidden email]> wrote:
> > > On 4/18/06, Chris Hostetter <[hidden email]> wrote:
> > > > (or does ServletContext.getServletContextName() not do what I think i
> > > > remember it doing)
> > >
> > > Unfortunately not.... the javadoc says it comes from the web.xml
> > >
> > >  java.lang.String       getServletContextName()
> > >           Returns the name of this web application corresponding to
> > > this ServletContext as specified in the deployment descriptor for this
> > > web application by the display-name element.
> > >
> > > -Yonik
> > >
> >
> >
> > --
> > Yoav Shapira
> > Nimalex LLC
> > 1 Mifflin Place, Suite 310
> > Cambridge, MA, USA
> > [hidden email] / www.yoavshapira.com
> >
>
>


--
Yoav Shapira
Nimalex LLC
1 Mifflin Place, Suite 310
Cambridge, MA, USA
[hidden email] / www.yoavshapira.com
Reply | Threaded
Open this post in threaded view
|

Re: multiple solr webapps

Yonik Seeley
In reply to this post by Yoav Shapira-2
On 4/18/06, Yoav Shapira <[hidden email]> wrote:
> There's no portable way of doing that, quite on purpose: the webapp is
> supposed to run the same independent of its server admin-dependent
> deployment details

Yeah, but it's not the same now; if I rename my webapp to solr1.war,
I'll be able to access it via the solr1 URL.  And when I get a
request, I can find out that someone used the solr1 path to get to me
(but that's to late for config type stuff that needs to happen in
init()).

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: multiple solr webapps

Yoav Shapira-2
Hola,

> request, I can find out that someone used the solr1 path to get to me
> (but that's to late for config type stuff that needs to happen in
> init()).

That but... clause is very important.  From what I understand, the
Servlet EG discussed this long and hard, many times, over many months,
before arriving at the current compromise.  There are also other
considerations (mod_rewrite comes to mind) that affects this
discussion.

Anyways, we're not going to change the spec, we're getting off-topic.
From my experience supporting Tomcat, approaches relying on the app's
context path or disk path are fragile at best.  The getResource
approaches (either ServletContext or classloader-based) are solid,
work well.  The JNDI approach is also solid and works well.  Other
approaches, like having a server-wide config file periodically
reloaded and used by all WARs, are also (marginally) OK.  None is a
zero-configuration approach, but all are fairly close, and easy for
both the developer to develop and the server admin to maintain.  We
could pick one, or we could develop our own thing...

Yoav
Reply | Threaded
Open this post in threaded view
|

Re: multiple solr webapps

Chris Hostetter-3
In reply to this post by Yoav Shapira-2

: Another approach to consider, then, is using the server's JNDI
: resources.  The code in the WAR would be the same, looking up the
: configuration file URL (or any other JNDI resource) using the
: java:comp/env space (which is different for every webapp, and portable
: across all J2EE servers/servlet containers), but the server's
: configuration file would specify different values for different
: webapps.  It's not zero configuration like just copying a WAR would
: be, but fairly close.

Just so I'm sure i'm understanding you:  You're saying that any java code
can uses JNDI to access data in the "java:comp/env/" namespace -- and that
the servlet spec requires that all application servers provide some way to
configure what lives in the "java:comp/env/" namespace on a per webapp
basis, independent of the webapp's web.xml (even if it doesn't mandate
what that mechanism is ... resin.conf in resin, server.xml in tomcat,
etc...)

So we could define a JNDI name like "java:comp/env/solr/home" and have
code like this...

    Context c = new InitialContext();
    File home = (File) c.lookup("java:comp/env/solr/home");
    if (null == home)
        home = new File(System.getProperty("solr.home","./solr"));
    }

...to determine the "root" directory for accessing all solr conig files
and data.

This would allow "advanced" users to configure which directory solr used
via JNDI -- even if they want multiple instances on a single server; or if
they only want one instance per server they could use a system property or
just use the default of "./solr" relative the current working directory
where they started the server.

If they were using Tomcat, then (from what i can tell) they would
configure their JNDI options in server.xml like this...

  <Context path="/solr-product-instance" >
    <Environment name="solr/home" value="/var/opt/solr/product"
                 type="java.lang.String" override="true"        />
  </Context>
  <Context path="/solr-article-instance" >
    <Environment name="solr/home" value="/var/opt/solr/articles"
                 type="java.lang.String" override="true"        />
  </Context>

...and just put two copies of solr.war in their webapps directlry: once
named solr-product-instance.war and once named solr-article-instance.war



Do I have that correct?


-Hoss

Reply | Threaded
Open this post in threaded view
|

Re: multiple solr webapps

Yoav Shapira-2
Hola,
<long snip />

> Do I have that correct?

Yeah.

--
Yoav Shapira
Nimalex LLC
1 Mifflin Place, Suite 310
Cambridge, MA, USA
[hidden email] / www.yoavshapira.com
Reply | Threaded
Open this post in threaded view
|

Re: multiple solr webapps

Bill Au
Using JNDI seems like a good approach.

We will need to add JNDI support as a requirement of the appserver for
running Solr.
Jetty doesn't support JNDI but JettyPlus does.

Bill

On 4/18/06, Yoav Shapira <[hidden email]> wrote:

>
> Hola,
> <long snip />
>
> > Do I have that correct?
>
> Yeah.
>
> --
> Yoav Shapira
> Nimalex LLC
> 1 Mifflin Place, Suite 310
> Cambridge, MA, USA
> [hidden email] / www.yoavshapira.com
>
Reply | Threaded
Open this post in threaded view
|

Re: multiple solr webapps

Yoav Shapira-2
I'd be shocked if Jetty (regular / free) doesn't support reading JNDI,
as I think that's a servlet spec requirement.  Some containers, like
Tomcat and apparently Jetty, make a full read/write JNDI
implementation not part of the core container.

Yoav

On 4/19/06, Bill Au <[hidden email]> wrote:

> Using JNDI seems like a good approach.
>
> We will need to add JNDI support as a requirement of the appserver for
> running Solr.
> Jetty doesn't support JNDI but JettyPlus does.
>
> Bill
>
> On 4/18/06, Yoav Shapira <[hidden email]> wrote:
> >
> > Hola,
> > <long snip />
> >
> > > Do I have that correct?
> >
> > Yeah.
> >
> > --
> > Yoav Shapira
> > Nimalex LLC
> > 1 Mifflin Place, Suite 310
> > Cambridge, MA, USA
> > [hidden email] / www.yoavshapira.com
> >
>
>


--
Yoav Shapira
Nimalex LLC
1 Mifflin Place, Suite 310
Cambridge, MA, USA
[hidden email] / www.yoavshapira.com
Reply | Threaded
Open this post in threaded view
|

Re: multiple solr webapps

Bill Au
It looks like we can use <env-entry>, which is part of the servlet specs:

<env-entry>
  <env-entry-name>solr/home</env-entry-name>
  <env-entry-type>java.lang.String</env-entry-type>
  <env-entry-value>/var/opt/solr/product</env-entry-value>
</env-entry>

The configuration will be the same for all containers.

Bill

On 4/19/06, Yoav Shapira <[hidden email]> wrote:

>
> I'd be shocked if Jetty (regular / free) doesn't support reading JNDI,
> as I think that's a servlet spec requirement.  Some containers, like
> Tomcat and apparently Jetty, make a full read/write JNDI
> implementation not part of the core container.
>
> Yoav
>
> On 4/19/06, Bill Au <[hidden email]> wrote:
> > Using JNDI seems like a good approach.
> >
> > We will need to add JNDI support as a requirement of the appserver for
> > running Solr.
> > Jetty doesn't support JNDI but JettyPlus does.
> >
> > Bill
> >
> > On 4/18/06, Yoav Shapira <[hidden email]> wrote:
> > >
> > > Hola,
> > > <long snip />
> > >
> > > > Do I have that correct?
> > >
> > > Yeah.
> > >
> > > --
> > > Yoav Shapira
> > > Nimalex LLC
> > > 1 Mifflin Place, Suite 310
> > > Cambridge, MA, USA
> > > [hidden email] / www.yoavshapira.com
> > >
> >
> >
>
>
> --
> Yoav Shapira
> Nimalex LLC
> 1 Mifflin Place, Suite 310
> Cambridge, MA, USA
> [hidden email] / www.yoavshapira.com
>
Reply | Threaded
Open this post in threaded view
|

Re: multiple solr webapps

Yoav Shapira-2
Yup, env-entry and resource-ref should both work.  That's what I meant
by a read-only JNDI impl, sorry if I was unclear...

Yoav

On 4/19/06, Bill Au <[hidden email]> wrote:

> It looks like we can use <env-entry>, which is part of the servlet specs:
>
> <env-entry>
>   <env-entry-name>solr/home</env-entry-name>
>   <env-entry-type>java.lang.String</env-entry-type>
>   <env-entry-value>/var/opt/solr/product</env-entry-value>
> </env-entry>
>
> The configuration will be the same for all containers.
>
> Bill
>
> On 4/19/06, Yoav Shapira <[hidden email]> wrote:
> >
> > I'd be shocked if Jetty (regular / free) doesn't support reading JNDI,
> > as I think that's a servlet spec requirement.  Some containers, like
> > Tomcat and apparently Jetty, make a full read/write JNDI
> > implementation not part of the core container.
> >
> > Yoav
> >
> > On 4/19/06, Bill Au <[hidden email]> wrote:
> > > Using JNDI seems like a good approach.
> > >
> > > We will need to add JNDI support as a requirement of the appserver for
> > > running Solr.
> > > Jetty doesn't support JNDI but JettyPlus does.
> > >
> > > Bill
> > >
> > > On 4/18/06, Yoav Shapira <[hidden email]> wrote:
> > > >
> > > > Hola,
> > > > <long snip />
> > > >
> > > > > Do I have that correct?
> > > >
> > > > Yeah.
> > > >
> > > > --
> > > > Yoav Shapira
> > > > Nimalex LLC
> > > > 1 Mifflin Place, Suite 310
> > > > Cambridge, MA, USA
> > > > [hidden email] / www.yoavshapira.com
> > > >
> > >
> > >
> >
> >
> > --
> > Yoav Shapira
> > Nimalex LLC
> > 1 Mifflin Place, Suite 310
> > Cambridge, MA, USA
> > [hidden email] / www.yoavshapira.com
> >
>
>


--
Yoav Shapira
Nimalex LLC
1 Mifflin Place, Suite 310
Cambridge, MA, USA
[hidden email] / www.yoavshapira.com