Multiple Solr Cores

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Multiple Solr Cores

Henrib-2
Following up on a previous thread in the Solr-User list, here is a patch that allows managing multiple cores in the same VM (thus multiple config/schemas/indexes).
The SolrCore.core singleton has been changed to a Map<String, SolrCore>; the current singleton behavior is keyed as 'null'. (Which is used by SolrInfoRegistry).
All static references to either a Config or a SolrCore have been removed; this implies that some classes now do refer to either a SolrCore or a SolrConfig (some ctors have been modified accordingly).

I haven't tried to modify anything above the 'jar' (script, admin & servlet are unaware of the multi-core part).

The 2 patches files are the src/ & the test/ patches.
solr-test.patch
solr-src.patch

This being my first attempt at a contribution, I will humbly welcome any comment.
Regards,
Henri
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Solr Cores

mpelzsherman
This sounds like a great idea, and potentially very useful for my company.

Can you explain a bit about how you would configure the various solr/home paths, and how the different indexes would be accessed by clients?

Thanks!

- Michael
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Solr Cores

Henrib-2
There is still only one solr.home instance used to load the various classes which is used as the one 'root'.
From there, you can have multiple solrconfig*.xml & schema*.xml (even absolute pathes); calling new SolrCore(name_of_core, path_to_solrconfig, path_to_schema) creates a named core that you can refer to.
To refer to a named core, you call SolrCore.getCore(name_of_core) (instead of SolrCore.getCore()).

From a servlet perspective, it seems that passing the name of the core back & forth should do the trick (so we can reacquire the correct core). One missing part is uploading a config & a schema then start a core (a dynamic creation of a core). One thing to note is that a schema needs a config to be created and it is certainly wise to use the same for schema & core creations.
For the admin servlet, we'd need to implement a way to choose the core we want to observe.
And the scripts probably also need to have a 'core name' passed down...

I'm still building my knowledge on the subject so my simplistic view might not be accurate.
Let me know if this helps.
Cheers
Henrib


mpelzsherman wrote
This sounds like a great idea, and potentially very useful for my company.

Can you explain a bit about how you would configure the various solr/home paths, and how the different indexes would be accessed by clients?

Thanks!

- Michael
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Solr Cores

Chris Hostetter-3
In reply to this post by Henrib-2

I'm sorry to say I am *way* behind on my patch reading (and moving into my
new place this weekend where i have no net access isn't going to help) so
i can't comment on the technique (or even style) of this patch ... but if
you could do peopel a favor and open a Jira issue and post it there for
people to review i would be very much appreciated...

http://wiki.apache.org/solr/HowToContribute
http://issues.apache.org/jira/browse/SOLR

...one quick comment based on something that jumped out at me from your
mail, removing public static methods will make it really hard to apply
this patch in a backwards compatible way (since many people may have
custom request handlers that rely on those methods) ... but if you're
using a "null" key in maps to refer to the default Core/Config you should
be able to keep the existing methods arround (deprecated) as wrappers for
your new methods right?


-Hoss

Reply | Threaded
Open this post in threaded view
|

Re: Multiple Solr Cores

Henrib-2
In reply to this post by Henrib-2

Updated (forgot the patch for Servlet).solr-trunk-src.patch

The change should still be compatible with the trunk it is based upon.

Henrib wrote
Following up on a previous thread in the Solr-User list, here is a patch that allows managing multiple cores in the same VM (thus multiple config/schemas/indexes).
The SolrCore.core singleton has been changed to a Map<String, SolrCore>; the current singleton behavior is keyed as 'null'. (Which is used by SolrInfoRegistry).
All static references to either a Config or a SolrCore have been removed; this implies that some classes now do refer to either a SolrCore or a SolrConfig (some ctors have been modified accordingly).

I haven't tried to modify anything above the 'jar' (script, admin & servlet are unaware of the multi-core part).

The 2 patches files are the src/ & the test/ patches.
solr-test.patch
solr-src.patch

This being my first attempt at a contribution, I will humbly welcome any comment.
Regards,
Henri
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Solr Cores

Jeff Kavanagh
You might want to have a look at this patch in case it breaks stuff for us

On 4/20/07, Henrib <[hidden email]> wrote:

>
>
> Updated (forgot the patch for Servlet).
> http://www.nabble.com/file/7996/solr-trunk-src.patch solr-trunk-src.patch
>
> The change should still be compatible with the trunk it is based upon.
>
>
> Henrib wrote:
> >
> > Following up on a previous thread in the Solr-User list, here is a patch
> > that allows managing multiple cores in the same VM (thus multiple
> > config/schemas/indexes).
> > The SolrCore.core singleton has been changed to a Map<String, SolrCore>;
> > the current singleton behavior is keyed as 'null'. (Which is used by
> > SolrInfoRegistry).
> > All static references to either a Config or a SolrCore have been removed;
> > this implies that some classes now do refer to either a SolrCore or a
> > SolrConfig (some ctors have been modified accordingly).
> >
> > I haven't tried to modify anything above the 'jar' (script, admin &
> > servlet are unaware of the multi-core part).
> >
> > The 2 patches files are the src/ & the test/ patches.
> >  http://www.nabble.com/file/7971/solr-test.patch solr-test.patch
> >  http://www.nabble.com/file/7972/solr-src.patch solr-src.patch
> >
> > This being my first attempt at a contribution, I will humbly welcome any
> > comment.
> > Regards,
> > Henri
> >
>
> --
> View this message in context: http://www.nabble.com/Multiple-Solr-Cores-tf3608399.html#a10106126
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Solr Cores

Jeff Kavanagh
Sorry everybody, please ignore my last message. Gmail protects from
accidental reply-alls, but not replies to mailing lists :).

(Thanks to the developers for all the fantastic work on Solr, btw.
Saving us a tonne of work.)

-Jeff


On 4/21/07, Jeff Kavanagh <[hidden email]> wrote:

> You might want to have a look at this patch in case it breaks stuff for us
>
> On 4/20/07, Henrib <[hidden email]> wrote:
> >
> >
> > Updated (forgot the patch for Servlet).
> > http://www.nabble.com/file/7996/solr-trunk-src.patch solr-trunk-src.patch
> >
> > The change should still be compatible with the trunk it is based upon.
> >
> >
> > Henrib wrote:
> > >
> > > Following up on a previous thread in the Solr-User list, here is a patch
> > > that allows managing multiple cores in the same VM (thus multiple
> > > config/schemas/indexes).
> > > The SolrCore.core singleton has been changed to a Map<String, SolrCore>;
> > > the current singleton behavior is keyed as 'null'. (Which is used by
> > > SolrInfoRegistry).
> > > All static references to either a Config or a SolrCore have been removed;
> > > this implies that some classes now do refer to either a SolrCore or a
> > > SolrConfig (some ctors have been modified accordingly).
> > >
> > > I haven't tried to modify anything above the 'jar' (script, admin &
> > > servlet are unaware of the multi-core part).
> > >
> > > The 2 patches files are the src/ & the test/ patches.
> > >  http://www.nabble.com/file/7971/solr-test.patch solr-test.patch
> > >  http://www.nabble.com/file/7972/solr-src.patch solr-src.patch
> > >
> > > This being my first attempt at a contribution, I will humbly welcome any
> > > comment.
> > > Regards,
> > > Henri
> > >
> >
> > --
> > View this message in context: http://www.nabble.com/Multiple-Solr-Cores-tf3608399.html#a10106126
> > Sent from the Solr - User mailing list archive at Nabble.com.
> >
> >
>