6

I manage many servers that span multiple environments (dev, qa, staging, and production). To help manage these, we have multiple repositories on a local webserver for our applications (e.g. app_1_el6, app_2_el7, etc.). We also mirror several upstream repos which provide dependencies for our custom rpms (e.g. EL Repo [1], EPEL [2], etc.) to reduce package download time.

Currently, a cron job syncs the our local with the upstream repositories. From time to time, we'll pull in an update that causes a problem (e.g. incompatibility with one of our packages). This ends up causing a lot of pain, which I'd like to avoid.

What'd I like to do is create some sort of version control for our local mirror of the upstream repos. I'd like to make sure, for instance, that if a new package is introduced in an upstream repo that breaks our custom rpms, that I have a way of rolling back or somehow isolating that package. What's the best way to go about this?

[1] http://elrepo.org/tiki/tiki-index.php

[2] https://fedoraproject.org/wiki/EPEL

CHK
  • 256
  • 2
  • 7

3 Answers3

4

Michael Hampton referenced an answer that names Katello and Spacewalk, Satellite is the product RedHat offers to accomplish that.

Katello is to Satellite what Fedora is to RedHat (according to this)

Lifecycle environments and content views are what you are looking for in Katello:

Promoting a Content View

Initially a Content View is published to Library as version 1. If there are Content Hosts in other environments that would like to consume this Content View, a version of the content view will need to be promoted to those environments. For example, given the Content View “New Content View”, version 1 of which has been promoted to the Dev environment. Any Content Hosts in Dev attached to the Content View would remain at version 1 until a version 2 is both published and promoted to the Dev environment.

Content view sample http://www.katello.org/docs/2.3/user_guide/content_views/promote_content_view2.png

Content view promoting progress http://www.katello.org/docs/2.3/user_guide/content_views/promote_content_view3.png

Community
  • 1
fuero
  • 9,413
  • 1
  • 35
  • 40
2

To expand on fuero's answer, we use RedHat Satellite to achieve this. Satellite is a combination of the open source tools Foreman and Katello. Specifically the Katello aspect is what provides this "Lifecycle Management".

In Katello, you define upstream repositories to synchronize from: either yum repos, puppet forges, git servers, etc. You then sync the content into the Library, and "promote" it through various environments. A common example would be "Library -> Dev -> Test -> Production".

There was a good talk at one of the Puppet conferences a while ago where some of the developers gave a demo of Katello/Foreman. Link on YouTube.

One thing to note: we're currently trying to solve the problem of binary distribution and tracking, which Katello doesn't solve. What I mean by that is that we have a set of Puppet modules and associated RPMs/binaries, but Katello provides no way to "snapshot" that for export to a different system. Katello will keep a static "view" of these things, but there's no way to confirm what's in that view - I can't say to the customer that they have "version X" of the system, nor can I confirm that the view they're using is exactly the same as mine. It all depends when they synchronized and what was in the repos at the time.

We're considering tools such as Nexus/Artifactory to provide this functionality at the moment, so you may want to have a look at those too.

shearn89
  • 3,143
  • 2
  • 14
  • 39
0

Well you could easily setup your own system to do that.
There is already a tool called reposync that you could use to sync entire repository.
Now the only missing link is how do you not clutter your diskspace with things.
Do a little bit readup on data deduplication with brtfs (for example as it was merged in the mainline kernel, check this project ) /You can use any other filesystem with data deduplication like Lessfs /
From there, you can setup your data storage space using any filesystem that allows deduplication and then use your cronjob to do the syncing, but this time timestamp the new sync, so lets say you can easily end up with this structure :
2016-05-15
2016-05-16
2017-05-17
production -> 2016-05-15 (symlinked)
staging -> 2016-05-17 (symlinked)
Now since you already have deduplication on that data, you won't be space starved.

Of course this adds up the overhead of having to maintain your own symlinking and what not, but hey using Katello you still have to click around :)

Community
  • 1
Hristo Mohamed
  • 1
  • 2