Add Solr files to VCS (GIT)

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Add Solr files to VCS (GIT)

Paras Lehana
Hi Community,

We have just set up a new server with Solr 8.4 on production. Instead of
changing files like solrconfig and solr.in.sh by logging on the server, we
are planning to have some VCS. We have integrated GIT on our server but, as
other servers, there is a single directory where git files are supposed to
be uploaded.

We have followed "Taking Solr to production" and thus our core conf files
live in /var/solr/data/core/conf while solr.in.sh is in /etc/default/.

What is your preferred method to integrate VCS for files like these? For a
workaround, I'm thinking of changing these files with symbolic links that
will point to git files. Or we can have an automated process (like rsync)
to copy files.

Just asking the community how they manage version control for Solr files.

--
--
Regards,

*Paras Lehana* [65871]
Development Engineer, Auto-Suggest,
IndiaMART Intermesh Ltd.

8th Floor, Tower A, Advant-Navis Business Park, Sector 142,
Noida, UP, IN - 201303

Mob.: +91-9560911996
Work: 01203916600 | Extn:  *8173*

--
*
*

 <https://www.facebook.com/IndiaMART/videos/578196442936091/>
Reply | Threaded
Open this post in threaded view
|

Re: Add Solr files to VCS (GIT)

Erick Erickson
There’s nothing built in to Solr that will automatically pull files from a
VCS repo, so it’s a manual process. Which is one of the “gotchas” about
managed config files, but that’s another rant.

Are you running SolrCloud or stand-alone? If SolrCloud, it doesn’t make
sense to talk about /var/solr/data/core/conf, you’d need to have one
copy in VCS and push/pull it to/from ZooKeeper as needed with
“bin/solr zk upconfig|downconfig”.

As for solr.in.sh, I’d keep a master copy in VCS and, if your setup is
complicated use something like Puppet or Chef or whatever to
automate pushing changes to all my servers. Otherwise do it
manually if you only have a few servers.

Best,
Erick


> On Jan 9, 2020, at 8:19 AM, Paras Lehana <[hidden email]> wrote:
>
> Hi Community,
>
> We have just set up a new server with Solr 8.4 on production. Instead of
> changing files like solrconfig and solr.in.sh by logging on the server, we
> are planning to have some VCS. We have integrated GIT on our server but, as
> other servers, there is a single directory where git files are supposed to
> be uploaded.
>
> We have followed "Taking Solr to production" and thus our core conf files
> live in /var/solr/data/core/conf while solr.in.sh is in /etc/default/.
>
> What is your preferred method to integrate VCS for files like these? For a
> workaround, I'm thinking of changing these files with symbolic links that
> will point to git files. Or we can have an automated process (like rsync)
> to copy files.
>
> Just asking the community how they manage version control for Solr files.
>
> --
> --
> Regards,
>
> *Paras Lehana* [65871]
> Development Engineer, Auto-Suggest,
> IndiaMART Intermesh Ltd.
>
> 8th Floor, Tower A, Advant-Navis Business Park, Sector 142,
> Noida, UP, IN - 201303
>
> Mob.: +91-9560911996
> Work: 01203916600 | Extn:  *8173*
>
> --
> *
> *
>
> <https://www.facebook.com/IndiaMART/videos/578196442936091/>

Reply | Threaded
Open this post in threaded view
|

Re: Add Solr files to VCS (GIT)

Paras Lehana
Hey Erick,

Thanks for your reply.

We have only one production server and one development server both
connected with same git repo (different branches). I'm using Solr in
standalone mode.

On Thu, 9 Jan 2020 at 19:48, Erick Erickson <[hidden email]> wrote:

> There’s nothing built in to Solr that will automatically pull files from a
> VCS repo, so it’s a manual process. Which is one of the “gotchas” about
> managed config files, but that’s another rant.
>
> Are you running SolrCloud or stand-alone? If SolrCloud, it doesn’t make
> sense to talk about /var/solr/data/core/conf, you’d need to have one
> copy in VCS and push/pull it to/from ZooKeeper as needed with
> “bin/solr zk upconfig|downconfig”.
>
> As for solr.in.sh, I’d keep a master copy in VCS and, if your setup is
> complicated use something like Puppet or Chef or whatever to
> automate pushing changes to all my servers. Otherwise do it
> manually if you only have a few servers.
>
> Best,
> Erick
>
>
> > On Jan 9, 2020, at 8:19 AM, Paras Lehana <[hidden email]>
> wrote:
> >
> > Hi Community,
> >
> > We have just set up a new server with Solr 8.4 on production. Instead of
> > changing files like solrconfig and solr.in.sh by logging on the server,
> we
> > are planning to have some VCS. We have integrated GIT on our server but,
> as
> > other servers, there is a single directory where git files are supposed
> to
> > be uploaded.
> >
> > We have followed "Taking Solr to production" and thus our core conf files
> > live in /var/solr/data/core/conf while solr.in.sh is in /etc/default/.
> >
> > What is your preferred method to integrate VCS for files like these? For
> a
> > workaround, I'm thinking of changing these files with symbolic links that
> > will point to git files. Or we can have an automated process (like rsync)
> > to copy files.
> >
> > Just asking the community how they manage version control for Solr files.
> >
> > --
> > --
> > Regards,
> >
> > *Paras Lehana* [65871]
> > Development Engineer, Auto-Suggest,
> > IndiaMART Intermesh Ltd.
> >
> > 8th Floor, Tower A, Advant-Navis Business Park, Sector 142,
> > Noida, UP, IN - 201303
> >
> > Mob.: +91-9560911996
> > Work: 01203916600 | Extn:  *8173*
> >
> > --
> > *
> > *
> >
> > <https://www.facebook.com/IndiaMART/videos/578196442936091/>
>
>

--
--
Regards,

*Paras Lehana* [65871]
Development Engineer, Auto-Suggest,
IndiaMART Intermesh Ltd.

8th Floor, Tower A, Advant-Navis Business Park, Sector 142,
Noida, UP, IN - 201303

Mob.: +91-9560911996
Work: 01203916600 | Extn:  *8173*

--
*
*

 <https://www.facebook.com/IndiaMART/videos/578196442936091/>
Reply | Threaded
Open this post in threaded view
|

Re: Add Solr files to VCS (GIT)

Walter Underwood
In reply to this post by Paras Lehana
For master/slave clusters, we have a deploy step that copies the config files
to each server. Then we restart the Solr process. We do that one at a time for
minimal service interruption.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)

> On Jan 9, 2020, at 5:19 AM, Paras Lehana <[hidden email]> wrote:
>
> Hi Community,
>
> We have just set up a new server with Solr 8.4 on production. Instead of
> changing files like solrconfig and solr.in.sh by logging on the server, we
> are planning to have some VCS. We have integrated GIT on our server but, as
> other servers, there is a single directory where git files are supposed to
> be uploaded.
>
> We have followed "Taking Solr to production" and thus our core conf files
> live in /var/solr/data/core/conf while solr.in.sh is in /etc/default/.
>
> What is your preferred method to integrate VCS for files like these? For a
> workaround, I'm thinking of changing these files with symbolic links that
> will point to git files. Or we can have an automated process (like rsync)
> to copy files.
>
> Just asking the community how they manage version control for Solr files.
>
> --
> --
> Regards,
>
> *Paras Lehana* [65871]
> Development Engineer, Auto-Suggest,
> IndiaMART Intermesh Ltd.
>
> 8th Floor, Tower A, Advant-Navis Business Park, Sector 142,
> Noida, UP, IN - 201303
>
> Mob.: +91-9560911996
> Work: 01203916600 | Extn:  *8173*
>
> --
> *
> *
>
> <https://www.facebook.com/IndiaMART/videos/578196442936091/>

Reply | Threaded
Open this post in threaded view
|

Re: Add Solr files to VCS (GIT)

Walter Underwood
In reply to this post by Paras Lehana
If you only have one server, that isn’t production or search isn’t important.

So it doesn’t really matter how you update it.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)

> On Jan 9, 2020, at 7:48 AM, Paras Lehana <[hidden email]> wrote:
>
> Hey Erick,
>
> Thanks for your reply.
>
> We have only one production server and one development server both
> connected with same git repo (different branches). I'm using Solr in
> standalone mode.
>
> On Thu, 9 Jan 2020 at 19:48, Erick Erickson <[hidden email]> wrote:
>
>> There’s nothing built in to Solr that will automatically pull files from a
>> VCS repo, so it’s a manual process. Which is one of the “gotchas” about
>> managed config files, but that’s another rant.
>>
>> Are you running SolrCloud or stand-alone? If SolrCloud, it doesn’t make
>> sense to talk about /var/solr/data/core/conf, you’d need to have one
>> copy in VCS and push/pull it to/from ZooKeeper as needed with
>> “bin/solr zk upconfig|downconfig”.
>>
>> As for solr.in.sh, I’d keep a master copy in VCS and, if your setup is
>> complicated use something like Puppet or Chef or whatever to
>> automate pushing changes to all my servers. Otherwise do it
>> manually if you only have a few servers.
>>
>> Best,
>> Erick
>>
>>
>>> On Jan 9, 2020, at 8:19 AM, Paras Lehana <[hidden email]>
>> wrote:
>>>
>>> Hi Community,
>>>
>>> We have just set up a new server with Solr 8.4 on production. Instead of
>>> changing files like solrconfig and solr.in.sh by logging on the server,
>> we
>>> are planning to have some VCS. We have integrated GIT on our server but,
>> as
>>> other servers, there is a single directory where git files are supposed
>> to
>>> be uploaded.
>>>
>>> We have followed "Taking Solr to production" and thus our core conf files
>>> live in /var/solr/data/core/conf while solr.in.sh is in /etc/default/.
>>>
>>> What is your preferred method to integrate VCS for files like these? For
>> a
>>> workaround, I'm thinking of changing these files with symbolic links that
>>> will point to git files. Or we can have an automated process (like rsync)
>>> to copy files.
>>>
>>> Just asking the community how they manage version control for Solr files.
>>>
>>> --
>>> --
>>> Regards,
>>>
>>> *Paras Lehana* [65871]
>>> Development Engineer, Auto-Suggest,
>>> IndiaMART Intermesh Ltd.
>>>
>>> 8th Floor, Tower A, Advant-Navis Business Park, Sector 142,
>>> Noida, UP, IN - 201303
>>>
>>> Mob.: +91-9560911996
>>> Work: 01203916600 | Extn:  *8173*
>>>
>>> --
>>> *
>>> *
>>>
>>> <https://www.facebook.com/IndiaMART/videos/578196442936091/>
>>
>>
>
> --
> --
> Regards,
>
> *Paras Lehana* [65871]
> Development Engineer, Auto-Suggest,
> IndiaMART Intermesh Ltd.
>
> 8th Floor, Tower A, Advant-Navis Business Park, Sector 142,
> Noida, UP, IN - 201303
>
> Mob.: +91-9560911996
> Work: 01203916600 | Extn:  *8173*
>
> --
> *
> *
>
> <https://www.facebook.com/IndiaMART/videos/578196442936091/>