small rsync index question

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

small rsync index question

Brian Whitman
I'm not using snap* scripts but i quickly need to sync up two indexes  
on two machines. I am rsyncing the data dirs from A to B, which work  
fine. But how can I see the new index on B? For some reason sending a  
<commit/> is not refreshing the index, and I have to restart resin to  
see it. Is there something else I have to do?

Reply | Threaded
Open this post in threaded view
|

Re: small rsync index question

Yonik Seeley-2
On 9/28/07, Brian Whitman <[hidden email]> wrote:
> For some reason sending a
> <commit/> is not refreshing the index

It should... are there any errors in the logs?  do you see the commit
in the logs?
Check the stats page to see info about when the current searcher was
last opened too.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: small rsync index question

Brian Whitman
  Sep 28, 2007, at 5:41 PM, Yonik Seeley wrote:
>
> It should... are there any errors in the logs?  do you see the commit
> in the logs?
> Check the stats page to see info about when the current searcher was
> last opened too.
>

ugh, nevermind.. was committing the wrong solr index... but Thanks  
yonik for the response

But luckily I can try save face with a followon question :)

I regularly see

file has vanished: "/dir/solr/data/index/segments_3aut"

when rsyncing, and when that happens i get an error on the rsync'd  
copy. The index I am rsyncing is large (50GB) and very active, is  
constantly getting new docs and searched on. What can I do to  
preserve the index state while syncing?





Reply | Threaded
Open this post in threaded view
|

Re: small rsync index question

Yonik Seeley-2
On 9/28/07, Brian Whitman <[hidden email]> wrote:
> I regularly see
>
> file has vanished: "/dir/solr/data/index/segments_3aut"
>
> when rsyncing, and when that happens i get an error on the rsync'd
> copy. The index I am rsyncing is large (50GB) and very active, is
> constantly getting new docs and searched on. What can I do to
> preserve the index state while syncing?

To greatly reduce this window, take a "snapshot" of the index (do it
like snapshooter, or actually use that script) and rsync from that
snapshot instead.

To completely remove the window of inconsistency, comment out the
post-commit hook in solrconfig.xml that takes a snapshot, then send a
commit to get a new snapshot and rsync from that.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: small rsync index question

hossman

: To completely remove the window of inconsistency, comment out the
: post-commit hook in solrconfig.xml that takes a snapshot, then send a
: commit to get a new snapshot and rsync from that.

i think yonik ment "UN-comment" the postCommit hook in the example
solrconfig.xml.


-Hoss