Replicating Lucene Index with out SOLR

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Replicating Lucene Index with out SOLR

rahul_k123
I have the following requirement

Right now we have multiple indexes  serving our web application. Our indexes are around 30 GB size.

We want to replicate the index data so that we can use them to distribute the search load.

This is what we need ideally.

A – (supports writes and reads)

A1 –Replicated Index (Supports reads)  . We want to synchronize this every 5 mins.



Any help is appreciated.   We are not using SOLR

I also interested in knowing what will be the best way so that I can scale my application adding more boxes for search if our load increases.

Thanks.  
Reply | Threaded
Open this post in threaded view
|

Re: Replicating Lucene Index with out SOLR

Otis Gospodnetic-2
Hi,

You may want to ask on the java-user list (more subscribers), which I'm CC-ing, so we can continue discussion there.
I think you will have to implement your own logic that runs on A and does something like this:

- stop adding new docs
- call commit on the IndexWriter

- copy the index
- resume indexing

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----

> From: rahul_k123 <[hidden email]>
> To: [hidden email]
> Sent: Thursday, August 28, 2008 1:34:41 AM
> Subject: Replicating Lucene Index with out SOLR
>
>
> I have the following requirement
>
> Right now we have multiple indexes  serving our web application. Our indexes
> are around 30 GB size.
>
> We want to replicate the index data so that we can use them to distribute
> the search load.
>
> This is what we need ideally.
>
> A – (supports writes and reads)
>
> A1 –Replicated Index (Supports reads)  . We want to synchronize this every 5
> mins.
>
>
>
> Any help is appreciated.   We are not using SOLR
>
> I also interested in knowing what will be the best way so that I can scale
> my application adding more boxes for search if our load increases.
>
> Thanks.  
>
> --
> View this message in context:
> http://www.nabble.com/Replicating-Lucene-Index-with-out-SOLR-tp19191752p19191752.html
> Sent from the Lucene - General mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Replicating Lucene Index with out SOLR

rahul_k123
Currently we index every certain amount of time on A.

-copy the index
     Copying the whole index everytime ?

Currently i am investigating how i can make use of SOLR replication scripts to achive this.


Is there anyone who did this with out SOLR before?


Thanks


Otis Gospodnetic wrote
Hi,

You may want to ask on the java-user list (more subscribers), which I'm CC-ing, so we can continue discussion there.
I think you will have to implement your own logic that runs on A and does something like this:

- stop adding new docs
- call commit on the IndexWriter

- copy the index
- resume indexing

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----
> From: rahul_k123 <vishnudeepak@gmail.com>
> To: general@lucene.apache.org
> Sent: Thursday, August 28, 2008 1:34:41 AM
> Subject: Replicating Lucene Index with out SOLR
>
>
> I have the following requirement
>
> Right now we have multiple indexes  serving our web application. Our indexes
> are around 30 GB size.
>
> We want to replicate the index data so that we can use them to distribute
> the search load.
>
> This is what we need ideally.
>
> A – (supports writes and reads)
>
> A1 –Replicated Index (Supports reads)  . We want to synchronize this every 5
> mins.
>
>
>
> Any help is appreciated.   We are not using SOLR
>
> I also interested in knowing what will be the best way so that I can scale
> my application adding more boxes for search if our load increases.
>
> Thanks.  
>
> --
> View this message in context:
> http://www.nabble.com/Replicating-Lucene-Index-with-out-SOLR-tp19191752p19191752.html
> Sent from the Lucene - General mailing list archive at Nabble.com.