Multiple Process of the SAME solr instance

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Multiple Process of the SAME solr instance

mohitranka
Hi All,
             I am using Solr 1.3 with Tomcat 5.5.20 as servlet container. I need to create multiple process of the same Solr instance to process the incoming indexes effectively. Can you point me how (and where :-) ) to do it?

Thanks and regards,
Mohit Ranka
Reply | Threaded
Open this post in threaded view
|

Re: [SPAM] Multiple Process of the SAME solr instance

Matthew Runo
I'm not 100% sure on what you mean, but if you're asking if you can  
run two or more solr webapps and use them all to build up one index,  
then you can't. You'll end up with a corrupted index. Only one  
solr.war webapp can write to an index at a time.

Thanks for your time!

Matthew Runo
Software Engineer, Zappos.com
[hidden email] - 702-943-7833

On Sep 17, 2008, at 7:54 AM, mohitranka wrote:

>
> Hi All,
>             I am using Solr 1.3 with Tomcat 5.5.20 as servlet  
> container. I
> need to create multiple process of the same Solr instance to process  
> the
> incoming indexes effectively. Can you point me how (and where :-) )  
> to do
> it?
>
> Thanks and regards,
> Mohit Ranka
>
> --
> View this message in context: http://www.nabble.com/Multiple-Process-of-the-SAME-solr-instance-tp19533951p19533951.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Reply | Threaded
Open this post in threaded view
|

Re: Multiple Process of the SAME solr instance

Otis Gospodnetic-2
In reply to this post by mohitranka
Hi Mohit,

I think we'll need a bit more info before we can help.  What kinds of processes do you need and what are you trying to achieve?

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----

> From: mohitranka <[hidden email]>
> To: [hidden email]
> Sent: Wednesday, September 17, 2008 10:54:09 AM
> Subject: Multiple Process of the SAME solr instance
>
>
> Hi All,
>              I am using Solr 1.3 with Tomcat 5.5.20 as servlet container. I
> need to create multiple process of the same Solr instance to process the
> incoming indexes effectively. Can you point me how (and where :-) ) to do
> it?
>
> Thanks and regards,
> Mohit Ranka
>
> --
> View this message in context:
> http://www.nabble.com/Multiple-Process-of-the-SAME-solr-instance-tp19533951p19533951.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Multiple Process of the SAME solr instance

mohitranka
In reply to this post by mohitranka
Thanks for your replies.
Actually the Solr instance will have many indexes to be updated simaltaneously, (say 100). Now i want to create 10 thread/process, so that I can process 10 indexes at a time, instead of 1.

I hope i am more clear with my requirement. :-)

Thanks and regards,
Mohit Ranka

mohitranka wrote
Hi All,
             I am using Solr 1.3 with Tomcat 5.5.20 as servlet container. I need to create multiple process of the same Solr instance to process the incoming indexes effectively. Can you point me how (and where :-) ) to do it?

Thanks and regards,
Mohit Ranka
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Process of the SAME solr instance

Otis Gospodnetic-2
In reply to this post by mohitranka
Mohit, it sounds like you are looking for http://wiki.apache.org/solr/MultipleIndexes

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----

> From: mohitranka <[hidden email]>
> To: [hidden email]
> Sent: Wednesday, September 17, 2008 3:14:38 PM
> Subject: Re: Multiple Process of the SAME solr instance
>
>
> Thanks for your replies.
> Actually the Solr instance will have many indexes to be updated
> simaltaneously, (say 100). Now i want to create 10 thread/process, so that I
> can process 10 indexes at a time, instead of 1.
>
> I hope i am more clear with my requirement. :-)
>
> Thanks and regards,
> Mohit Ranka
>
>
> mohitranka wrote:
> >
> > Hi All,
> >              I am using Solr 1.3 with Tomcat 5.5.20 as servlet container.
> > I need to create multiple process of the same Solr instance to process the
> > incoming indexes effectively. Can you point me how (and where :-) ) to do
> > it?
> >
> > Thanks and regards,
> > Mohit Ranka
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Multiple-Process-of-the-SAME-solr-instance-tp19533951p19539147.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Multiple Process of the SAME solr instance

mohitranka
Otis,
        Thanks for your reply. I think i misdirected you from my previous message. What I meant was 100 "documents". which should be added to solr index.  

Sorry for lack of clarity in the query.

Thanks and regards,
Mohit Ranla

Otis Gospodnetic wrote
Mohit, it sounds like you are looking for http://wiki.apache.org/solr/MultipleIndexes

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----
> From: mohitranka <mohitranka@gmail.com>
> To: solr-user@lucene.apache.org
> Sent: Wednesday, September 17, 2008 3:14:38 PM
> Subject: Re: Multiple Process of the SAME solr instance
>
>
> Thanks for your replies.
> Actually the Solr instance will have many indexes to be updated
> simaltaneously, (say 100). Now i want to create 10 thread/process, so that I
> can process 10 indexes at a time, instead of 1.
>
> I hope i am more clear with my requirement. :-)
>
> Thanks and regards,
> Mohit Ranka
>
>
> mohitranka wrote:
> >
> > Hi All,
> >              I am using Solr 1.3 with Tomcat 5.5.20 as servlet container.
> > I need to create multiple process of the same Solr instance to process the
> > incoming indexes effectively. Can you point me how (and where :-) ) to do
> > it?
> >
> > Thanks and regards,
> > Mohit Ranka
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Multiple-Process-of-the-SAME-solr-instance-tp19533951p19539147.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Process of the SAME solr instance

Otis Gospodnetic-2
In reply to this post by mohitranka
Mohit,

Have you tried following the Solr tutorial?  Adding multiple documents to Solr is a normal Solr usage and you go through that if you follow the tutorial on the site.

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----

> From: mohitranka <[hidden email]>
> To: [hidden email]
> Sent: Wednesday, September 17, 2008 4:33:26 PM
> Subject: Re: Multiple Process of the SAME solr instance
>
>
> Otis,
>         Thanks for your reply. I think i misdirected you from my previous
> message. What I meant was 100 "documents". which should be added to solr
> index.  
>
> Sorry for lack of clarity in the query.
>
> Thanks and regards,
> Mohit Ranla
>
>
> Otis Gospodnetic wrote:
> >
> > Mohit, it sounds like you are looking for
> > http://wiki.apache.org/solr/MultipleIndexes
> >
> > Otis
> > --
> > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
> >
> >
> >
> > ----- Original Message ----
> >> From: mohitranka
> >> To: [hidden email]
> >> Sent: Wednesday, September 17, 2008 3:14:38 PM
> >> Subject: Re: Multiple Process of the SAME solr instance
> >>
> >>
> >> Thanks for your replies.
> >> Actually the Solr instance will have many indexes to be updated
> >> simaltaneously, (say 100). Now i want to create 10 thread/process, so
> >> that I
> >> can process 10 indexes at a time, instead of 1.
> >>
> >> I hope i am more clear with my requirement. :-)
> >>
> >> Thanks and regards,
> >> Mohit Ranka
> >>
> >>
> >> mohitranka wrote:
> >> >
> >> > Hi All,
> >> >              I am using Solr 1.3 with Tomcat 5.5.20 as servlet
> >> container.
> >> > I need to create multiple process of the same Solr instance to process
> >> the
> >> > incoming indexes effectively. Can you point me how (and where :-) ) to
> >> do
> >> > it?
> >> >
> >> > Thanks and regards,
> >> > Mohit Ranka
> >> >
> >> >
> >>
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/Multiple-Process-of-the-SAME-solr-instance-tp19533951p19539147.html
> >> Sent from the Solr - User mailing list archive at Nabble.com.
> >
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Multiple-Process-of-the-SAME-solr-instance-tp19533951p19540549.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Multiple Process of the SAME solr instance

mohitranka
Otis, I understand that 1 solr instance can store n documents (one-by-one). My query was how to create m such instances/processes/threads so that m documents get stored at a time, instead of 1 at a time.

All the instances should read at the same port.

Otis Gospodnetic wrote
Mohit,

Have you tried following the Solr tutorial?  Adding multiple documents to Solr is a normal Solr usage and you go through that if you follow the tutorial on the site.

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----
> From: mohitranka <mohitranka@gmail.com>
> To: solr-user@lucene.apache.org
> Sent: Wednesday, September 17, 2008 4:33:26 PM
> Subject: Re: Multiple Process of the SAME solr instance
>
>
> Otis,
>         Thanks for your reply. I think i misdirected you from my previous
> message. What I meant was 100 "documents". which should be added to solr
> index.  
>
> Sorry for lack of clarity in the query.
>
> Thanks and regards,
> Mohit Ranla
>
>
> Otis Gospodnetic wrote:
> >
> > Mohit, it sounds like you are looking for
> > http://wiki.apache.org/solr/MultipleIndexes
> >
> > Otis
> > --
> > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
> >
> >
> >
> > ----- Original Message ----
> >> From: mohitranka
> >> To: solr-user@lucene.apache.org
> >> Sent: Wednesday, September 17, 2008 3:14:38 PM
> >> Subject: Re: Multiple Process of the SAME solr instance
> >>
> >>
> >> Thanks for your replies.
> >> Actually the Solr instance will have many indexes to be updated
> >> simaltaneously, (say 100). Now i want to create 10 thread/process, so
> >> that I
> >> can process 10 indexes at a time, instead of 1.
> >>
> >> I hope i am more clear with my requirement. :-)
> >>
> >> Thanks and regards,
> >> Mohit Ranka
> >>
> >>
> >> mohitranka wrote:
> >> >
> >> > Hi All,
> >> >              I am using Solr 1.3 with Tomcat 5.5.20 as servlet
> >> container.
> >> > I need to create multiple process of the same Solr instance to process
> >> the
> >> > incoming indexes effectively. Can you point me how (and where :-) ) to
> >> do
> >> > it?
> >> >
> >> > Thanks and regards,
> >> > Mohit Ranka
> >> >
> >> >
> >>
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/Multiple-Process-of-the-SAME-solr-instance-tp19533951p19539147.html
> >> Sent from the Solr - User mailing list archive at Nabble.com.
> >
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Multiple-Process-of-the-SAME-solr-instance-tp19533951p19540549.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Process of the SAME solr instance

mohitranka
Will having multiple cores, instead of one, server the purpose?
mohitranka wrote
Otis, I understand that 1 solr instance can store n documents (one-by-one). My query was how to create m such instances/processes/threads so that m documents get stored at a time, instead of 1 at a time.

All the instances should read at the same port.

Otis Gospodnetic wrote
Mohit,

Have you tried following the Solr tutorial?  Adding multiple documents to Solr is a normal Solr usage and you go through that if you follow the tutorial on the site.

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----
> From: mohitranka <mohitranka@gmail.com>
> To: solr-user@lucene.apache.org
> Sent: Wednesday, September 17, 2008 4:33:26 PM
> Subject: Re: Multiple Process of the SAME solr instance
>
>
> Otis,
>         Thanks for your reply. I think i misdirected you from my previous
> message. What I meant was 100 "documents". which should be added to solr
> index.  
>
> Sorry for lack of clarity in the query.
>
> Thanks and regards,
> Mohit Ranla
>
>
> Otis Gospodnetic wrote:
> >
> > Mohit, it sounds like you are looking for
> > http://wiki.apache.org/solr/MultipleIndexes
> >
> > Otis
> > --
> > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
> >
> >
> >
> > ----- Original Message ----
> >> From: mohitranka
> >> To: solr-user@lucene.apache.org
> >> Sent: Wednesday, September 17, 2008 3:14:38 PM
> >> Subject: Re: Multiple Process of the SAME solr instance
> >>
> >>
> >> Thanks for your replies.
> >> Actually the Solr instance will have many indexes to be updated
> >> simaltaneously, (say 100). Now i want to create 10 thread/process, so
> >> that I
> >> can process 10 indexes at a time, instead of 1.
> >>
> >> I hope i am more clear with my requirement. :-)
> >>
> >> Thanks and regards,
> >> Mohit Ranka
> >>
> >>
> >> mohitranka wrote:
> >> >
> >> > Hi All,
> >> >              I am using Solr 1.3 with Tomcat 5.5.20 as servlet
> >> container.
> >> > I need to create multiple process of the same Solr instance to process
> >> the
> >> > incoming indexes effectively. Can you point me how (and where :-) ) to
> >> do
> >> > it?
> >> >
> >> > Thanks and regards,
> >> > Mohit Ranka
> >> >
> >> >
> >>
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/Multiple-Process-of-the-SAME-solr-instance-tp19533951p19539147.html
> >> Sent from the Solr - User mailing list archive at Nabble.com.
> >
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Multiple-Process-of-the-SAME-solr-instance-tp19533951p19540549.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Process of the SAME solr instance

Shalin Shekhar Mangar
In reply to this post by mohitranka
On Thu, Sep 18, 2008 at 11:03 AM, mohitranka <[hidden email]> wrote:

>
> Otis, I understand that 1 solr instance can store n documents (one-by-one).
> My query was how to create m such instances/processes/threads so that m
> documents get stored at a time, instead of 1 at a time.
>
> All the instances should read at the same port.
>
>
You can send a batch of m documents at a time in the same XML.

--
Regards,
Shalin Shekhar Mangar.
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Process of the SAME solr instance

mohitranka
Shalin,
            I understand that :-)

My problem is, if 1 solr instance process(save) 100 documents one-by-one, it would not be very effective, I want to create 10 clones (process/threads/cores) of the same solr instance, so that 10 documents get processed(saved to solr) simaltaneously.

Thanks and regards,
Mohit Ranka
     
Shalin Shekhar Mangar wrote
On Thu, Sep 18, 2008 at 11:03 AM, mohitranka <mohitranka@gmail.com> wrote:

>
> Otis, I understand that 1 solr instance can store n documents (one-by-one).
> My query was how to create m such instances/processes/threads so that m
> documents get stored at a time, instead of 1 at a time.
>
> All the instances should read at the same port.
>
>
You can send a batch of m documents at a time in the same XML.

--
Regards,
Shalin Shekhar Mangar.
Reply | Threaded
Open this post in threaded view
|

Re: Multiple Process of the SAME solr instance

Otis Gospodnetic-2
In reply to this post by mohitranka
Mohit,

I think you are thinking too hard - trying to optimize something that doesn't sound like it needs optimizing at this point in your project.  I suggest you start with 1 Solr instance and then see if anything needs to be faster after you've pushed that to its limits.

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----

> From: mohitranka <[hidden email]>
> To: [hidden email]
> Sent: Thursday, September 18, 2008 2:15:25 AM
> Subject: Re: Multiple Process of the SAME solr instance
>
>
> Shalin,
>             I understand that :-)
>
> My problem is, if 1 solr instance process(save) 100 documents one-by-one, it
> would not be very effective, I want to create 10 clones
> (process/threads/cores) of the same solr instance, so that 10 documents get
> processed(saved to solr) simaltaneously.
>
> Thanks and regards,
> Mohit Ranka
>      
>
> Shalin Shekhar Mangar wrote:
> >
> > On Thu, Sep 18, 2008 at 11:03 AM, mohitranka wrote:
> >
> >>
> >> Otis, I understand that 1 solr instance can store n documents
> >> (one-by-one).
> >> My query was how to create m such instances/processes/threads so that m
> >> documents get stored at a time, instead of 1 at a time.
> >>
> >> All the instances should read at the same port.
> >>
> >>
> > You can send a batch of m documents at a time in the same XML.
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Multiple-Process-of-the-SAME-solr-instance-tp19533951p19546626.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Multiple Process of the SAME solr instance

hossman
In reply to this post by mohitranka

: My problem is, if 1 solr instance process(save) 100 documents one-by-one, it
: would not be very effective, I want to create 10 clones
: (process/threads/cores) of the same solr instance, so that 10 documents get
: processed(saved to solr) simaltaneously.

Perhaps i'm completley missunderstanding you, but it doesn't sound like
your concern is about having more then one *solr* instance or process, or
index ... it sounds like your concern is in indexing documents one a time
in serial, and you would like to parallelize.  Solr (as a server) cna
already handle this, you just need to parallelize your (client) indexing
code.



-Hoss