21

A new RedHat EL5.6 server has been recently put under my care. It is immediately obvious that, for the previous 12 months, little to no attention has been given to any sort of package updates.

Typically I am of the mindset of if it isn't broke - don't fix it. However, after registering the server with RHN, and also using the yum-security plugin to check security updates, there are just over 1100 "security" updates available.

Has anybody had a similar situation? I am reluctant to just update everything, as I like to know what is being updated and whether or not it has the potential to impact anything running on the box (this is a production server). However, it also looks like keeping in line with this practice would require me to go through 1100 package errata line by line. Is there a more efficient solution?

tdk2fe
  • 600
  • 2
  • 13

2 Answers2

21

Generally speaking speaking security updates are considered to be somewhat safe, particularly for a distribution with goals like RedHat. Their core focus is creating an operating environment that is consistent. As such the maintainers tend to pick versions of packages and stick with them for the long haul. To see what I mean look at the versions of such packages like kernel, python, perl, and httpd. What they also do is backport security patches from the upstream developers. So if a security vulnerability is found for all versions of Apache httpd 2.2.x then the Apache foundation may release version 2.2.40 with the fix, but RedHat will roll the patch locally and release httpd-2.2.3-80 with the fix.

Also keep in mind that you're currently talking about a RHEL5.7 system, the current release is 5.9. Some software vendors will only support certain subreleases. I've recently come across one piece of software, for example, that the vendor says only works on 5.4. That doesn't mean it won't run on 5.9, but it may mean that they won't provide any support it if doesn't work.

There are also concerns with doing mass updates of a system that hasn't been patched in such a long time. The biggest one that I've come across is actually more of a configuration management problem that can just be exacerbated by big updates. Sometimes a config file is changed but the administrator never restarts the service. This means that the config on disk has never been tested, and the running config may no longer exist. So if the service gets restarted, which will happen once you apply the kernel updates, it may not actually restart. Or it may act different once it restarts.

My advice, would be to do the updates, but be smart about it.

  • Plan it out during a maintenance window. If nothing else the server will require restarting, there have been a number of kernel updates and you will have to reboot to apply them.
  • Make sure to take a full backup before doing anything. This could be snapshotting, if this is a VM, triggering a full backup on whatever your tool is, tarring up / (to another system), taking a dd image of the drives, whatever. Just so long as it's something you can restore from.
  • Plan out how you apply the updates. You don't want to just throw a yum update -y at it and walk away. For all of the good things that yum does do it does not order when it applies updates according to the dependencies. This has caused problems in the past. I always run yum clean all && yum update -y yum && yum update -y glibc && yum update. That tends to take care of most of the potential ordering issues.

This may also be a great time to replatform. We've had RHEL6 for quite a while now. Depending on what this server does, it may make sense to just let this one run as is while you bring up a new instance in parallel. Then once it's installed you can copy all the data over, test the services, and perform the cut over. This will also give you the chance to know, from the ground up, that the system is standardized, clean, well documented, and all that jazz.

No matter what you do, I feel it's pretty important that you get yourself up to a current system. You just need to make sure to do it in a way that lets you trust your work and the finished product.

Scott Pack
  • 14,717
  • 10
  • 51
  • 83
  • Thank you for the detailed reply and insight on some considerations I haven't thought of. Hopefully this will be the last bulk update I need to do. As I mentioned in my post, they weren't even using RHN (which they purchased) so getting the application owners on board will likely be more difficult than the technical work :) – tdk2fe Mar 04 '13 at 14:52
  • @tdk2fe: It always is. :) Honestly, depending on what this thing runs, it should be pretty stinking stable. I would be much less concerned with applications not working anymore than I would services actually starting back up. – Scott Pack Mar 04 '13 at 14:56
  • Don't be too concerned about RHEL 5.7 vs 5.9, RHEL 5.9 is just RHEL 5.0 with all updates since, shipped ready to install. There really isn't a need to "upgrade" inside the series. I'd advise to _at least_ install the security updates, and seriously consider installing the rest as applicable. Can't you set up a virtual machine, or a test box, in which to replicate the main machine and check out nothing explodes too badly? – vonbrand Mar 04 '13 at 15:00
  • 1
    @vonbrand: Right, exactly. The point releases are effectively just tag cuts of the repo at specific dates. That doesn't mean there *won't* be problems. The gilbc kerfuffle from 5.3 to 5.4 is a fantastic example. – Scott Pack Mar 04 '13 at 15:12
  • @tdk2fe Not to mention, if the system was _only_ unmaintained for a year, you're doing pretty well. Most of us have seen systems go without attention for several years... – Michael Hampton Mar 04 '13 at 21:04
  • @MichaelHampton: I can't help but think of the time I saw a machine with a greater than 10 year uptime. – Scott Pack Mar 04 '13 at 21:06
3

In my experience RHEL does not break backwards compatibility in same-release updates.

that, however will not extend to anything that has been installed externally to rpm.

You could use rpm -qf to find the files you suspect are compiled external, if it returns "not owned by any package" then you might have issues in your upgrade.

I'd take an image of the server and do the upgrade, I'm a little more devil-may-care than most however.

dijit
  • 31
  • 1
  • Good point. Check if `/etc/yum.repos.d` has some strange repositories configured ([EPEL](http://fedoraproject.org/wiki/EPEL) should be safe), install `yum-utils` and check the output of `package-cleanup --orphans` (packages installed that aren't in any configured repository). – vonbrand Mar 04 '13 at 15:03