CDCR "all" collections

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

CDCR "all" collections

Bram Van Dam
Hey folks,

Is there any way to set up CDCR for *all* collections, including any
newly created ones? Having to modify the solrconfig in ZK every time a
collection is added is a bit of a pain, especially because I'm assuming
it requires a restart to activate the config?

Basically if I have DC Src and DC Tgt, I want every collection from Src
to be replicated to Tgt. Even when I create a new collection on Src.

Thanks,

 - Bram
Reply | Threaded
Open this post in threaded view
|

Re: CDCR "all" collections

Erick Erickson
Bram:

Hmmm You can't do that OOB right now, but it might not be a hard thing to add.

The current configuration allows the source collection to have a
different name than the
target collection so if you could make the assumption that the two
collections always had
the same name, it might be trivial.

WARNING! this is something that just occurred to me. I have NOT
thought it through,
but if it works it'd be very cool ;)

How brave do you feel? This _might_ be totally trivial. I'm looking at
the current trunk, but
in CdcrReplicationManager, line 97 looks like this:

String targetCollection = params.get(CdcrParams.TARGET_COLLECTION_PARAM);

It _might_ (and again, I have NOT explored this in detail) be as
simple as adding
after that line:

if (targetCollection == null) {
    targetCollection = params.get(CdcrParams.SOURCE_COLLECTION_PARAM);
}

or similar. Then leave

<str name="target">collection1</str>

out of the solrconfig file.

While the code change is trivial, the work is in verifying that it
works and I'm afraid
I don't personally have the time to do that verification, but I'd be
glad to commit if
if someone else does and submits a patch, including at least one unit test.

The tricky parts would be insuring nothing bad happens if, for
instance, the target
collection never got created, making sure the tlogs didn't grow, that
kind of thing.

Best,
Erick

On Thu, Jan 24, 2019 at 3:51 AM Bram Van Dam <[hidden email]> wrote:

>
> Hey folks,
>
> Is there any way to set up CDCR for *all* collections, including any
> newly created ones? Having to modify the solrconfig in ZK every time a
> collection is added is a bit of a pain, especially because I'm assuming
> it requires a restart to activate the config?
>
> Basically if I have DC Src and DC Tgt, I want every collection from Src
> to be replicated to Tgt. Even when I create a new collection on Src.
>
> Thanks,
>
>  - Bram