Re: [jira] Commented: (SOLR 2 1 5) Multiple Solr Cores

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (SOLR 2 1 5) Multiple Solr Cores

Yonik Seeley-2
On 7/11/07, Will Johnson <[hidden email]> wrote:
> I think it would be nice to have the core name
> specified as a CGI param instead of (or in addition to) a url path.
> Otherwise, large section of client code (such as solrj/solr#) will need
> to be changed.

Only if you want to talk to multiple cores over a single "connection", right?
Hopefully existing client code will allow the specification of the URL, and
one would use http://localhost:8983/solr/core1/

Still might be useful as a param *if* it can be done efficiently.
I wonder about the case when the param comes in via POST though.

-Yonik
Reply | Threaded
Open this post in threaded view
|

RE: [jira] Commented: (SOLR 2 1 5) Multiple Solr Cores

Will Johnson
Most of the time I, and I imagine others, don't know the set of core's
ahead of time.  It seems somewhat wasteful to create a ton of solr
server connections when a single one can handle things just as easily.
I guess I don't see why this param should be any different than any
others like output formats etc.    

As for POST's, you can still have cgi arguments and access them via the
same servlet request parameters while accessing the input stream.

I'll leave the efficiency issues to people more familiar with the patch
but if it has to be in the url then you force people using solrj and
other similar apis to create a Map<String, SolrServer> and manage them
that way.

- will

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Yonik
Seeley
Sent: Wednesday, July 11, 2007 1:20 PM
To: [hidden email]
Subject: Re: [jira] Commented: (SOLR 2 1 5) Multiple Solr Cores

On 7/11/07, Will Johnson <[hidden email]> wrote:
> I think it would be nice to have the core name
> specified as a CGI param instead of (or in addition to) a url path.
> Otherwise, large section of client code (such as solrj/solr#) will
need
> to be changed.

Only if you want to talk to multiple cores over a single "connection",
right?
Hopefully existing client code will allow the specification of the URL,
and
one would use http://localhost:8983/solr/core1/

Still might be useful as a param *if* it can be done efficiently.
I wonder about the case when the param comes in via POST though.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (SOLR 2 1 5) Multiple Solr Cores

Henrib-2
In reply to this post by Yonik Seeley-2
Yonik Seeley wrote
On 7/11/07, Will Johnson <wjohnson@getconnected.com> wrote:
> I think it would be nice to have the core name
> specified as a CGI param instead of (or in addition to) a url path.
> Otherwise, large section of client code (such as solrj/solr#) will need
> to be changed.

Only if you want to talk to multiple cores over a single "connection", right?
Hopefully existing client code will allow the specification of the URL, and
one would use http://localhost:8983/solr/core1/

Still might be useful as a param *if* it can be done efficiently.
I wonder about the case when the param comes in via POST though.

-Yonik
Passing the core name as an Http request parameter to use an existing core is easy.
In the patched SolrServlet, replacing the first few lines of doGet with the code that follows should do the trick.
-----------------
 
  private SolrCore getParameterCore(HttpServletRequest request) {
    SolrCore core = null;
    String[] score = request.getParameterValues("core");
    if (score == null)
      return null;
    // lets try till we find one valid core
    for(int c = 0; c < score.length && core == null; ++c)
      core = SolrCore.getSolrCore(score[c]);
    return core;
  }
 
  public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
    SolrCore core = getParameterCore(request);
    if (core == null) {
      if (this.core == null) {
        Object ocore = request.getAttribute("org.apache.solr.SolrCore");
        if (ocore instanceof SolrCore)
          core = (SolrCore) core;
        else
          sendErr(404, "no core to serve request", request, response);
      }
      else
        core = this.core;
    }
...
-------------------

Creating a core dynamically needs a bit more work since we then need to pass the config & schema to use; suggested ways were naming convention or/and even potentially upload them.
Hope this helps.
- Henri
Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (SOLR 2 1 5) Multiple Solr Cores

Yonik Seeley-2
On 7/11/07, Henrib <[hidden email]> wrote:
> Passing the core name as an Http request parameter to use an existing core
> is easy.
> In the patched SolrServlet, replacing the first few lines of doGet with the
> code that follows should do the trick.

Not quite that easy, given that we are using a filter now (SolrDispatchFilter),
which uses SolrRequestParsers to get the parameters (which currently
requires the core), and can do other things such as handle a binary
post body in addition to parsing params from the URL.  It looks like
some of that code should be refactored so that the core is not needed,
and then the core parameter could be retrieved from the resulting
SolrParams.

IMO SolrServlet, being the older (original) implementation need not
support multiple cores.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (SOLR 2 1 5) Multiple Solr Cores

Henrib-2
You're absolutely right. I went too fast.
We can access the http request parameters from the servlet filter though, so this "quick fix" might be possible.
Henri

Yonik Seeley wrote
On 7/11/07, Henrib <hbiestro@gmail.com> wrote:
> Passing the core name as an Http request parameter to use an existing core
> is easy.
> In the patched SolrServlet, replacing the first few lines of doGet with the
> code that follows should do the trick.

Not quite that easy, given that we are using a filter now (SolrDispatchFilter),
which uses SolrRequestParsers to get the parameters (which currently
requires the core), and can do other things such as handle a binary
post body in addition to parsing params from the URL.  It looks like
some of that code should be refactored so that the core is not needed,
and then the core parameter could be retrieved from the resulting
SolrParams.

IMO SolrServlet, being the older (original) implementation need not
support multiple cores.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (SOLR 2 1 5) Multiple Solr Cores

Yonik Seeley-2
On 7/11/07, Henrib <[hidden email]> wrote:
> You're absolutely right. I went too fast.
> We can access the http request parameters from the servlet filter though, so
> this "quick fix" might be possible.
> Henri

I'm not sure if that would handle params as part of a multi-part
upload (which uses a 3rd party lib since it's not supported by servlet
containers AFAIK).

-Yonik

>
> Yonik Seeley wrote:
> >
> > On 7/11/07, Henrib <[hidden email]> wrote:
> >> Passing the core name as an Http request parameter to use an existing
> >> core
> >> is easy.
> >> In the patched SolrServlet, replacing the first few lines of doGet with
> >> the
> >> code that follows should do the trick.
> >
> > Not quite that easy, given that we are using a filter now
> > (SolrDispatchFilter),
> > which uses SolrRequestParsers to get the parameters (which currently
> > requires the core), and can do other things such as handle a binary
> > post body in addition to parsing params from the URL.  It looks like
> > some of that code should be refactored so that the core is not needed,
> > and then the core parameter could be retrieved from the resulting
> > SolrParams.
> >
> > IMO SolrServlet, being the older (original) implementation need not
> > support multiple cores.
> >
> > -Yonik
> >
> >
>
> --
> View this message in context: http://www.nabble.com/Re%3A--jira--Commented%3A-%28SOLR-2-1-5%29-Multiple-Solr-Cores-tf4063316.html#a11548596
> Sent from the Solr - Dev mailing list archive at Nabble.com.
>
>