20

I have a few production Fedora and Debian webservers that host our sites as well as user shell accounts (used for git vcs work, some screen+irssi sessions, etc).

Occasionally a new kernel update will come down the pipeline in yum/apt-get, and I was wondering if most of the fixes are severe enough to warrant a reboot, or if I can apply the fixes sans reboot.

Our main development server currently has 213 days of uptime, and I wasn't sure if it was insecure to run such an older kernel.

poolie
  • 1,155
  • 1
  • 9
  • 17
lfaraone
  • 1,581
  • 2
  • 17
  • 22
  • You should really separate production hosting (fairly exposed and security-critical, should get updates right away) from running git repos (probably only trusted users but must be secure) and general screen sessions. VMs are cheap! – poolie May 27 '14 at 08:57

8 Answers8

24

There is nothing really special about having a long uptime. It is generally better to have a secure system. All systems need updates at some point. You are probably already applying updates, do you schedule outages when you apply those updates? You probably should just in case something goes wrong. A reboot shouldn't that that much time really.

If your system is so sensitive to outages, you probably should be thinking about some kind of clustering setup so you update a single member of the cluster without bringing everything down.

If you are not sure about a particular update it is probably safer to schedule a reboot and apply it (preferably after testing it on another similar system).

If you are interested in learning about if the update is important take time to read the security notice, and follow the links back to the CVE or the posts/lists/blogs describing the issue. This should help you decide if the update directly applies in your case.

Even if you don't think it applies you should still consider updating your system eventually. Security is a layered approach. You should assume at some point in time those other layers may fail. Also, you might forget you have a vulnerable system because you skipped an update when you change the configuration at some later point in time.

Anyway if you want to ignore or wait for a while on update on Debian based systems you can put the package on hold. I personally like to put holds on all the kernel packages just in case.

CLI method to set a hold on a package on Debian-based systems.

dpkg --get-selections | grep 'linux-image' | sed -e 's/install/hold/' | sudo dpkg --set-selections
Zoredache
  • 128,755
  • 40
  • 271
  • 413
  • 1
    It's not that we need to be always-on, but rather that some of our users have open sessions (i.e. IRC) that can be annoying (from the user's perspective) to restart. – lfaraone May 19 '09 at 23:20
13

Most updates do not require a reboot, but Kernel updates do (you can't really replace the running kernel without rebooting).

One thing I have discovered is that if your server has been running for a long time without a reboot, it's more likely to want to do disk checks (fsck) when you reboot, and this can add significantly to the time it takes to get back up and running again. Best to anticipate this and plan for it.

I have also discovered that configuration changes can sometimes get missed, and won't be noticed until a reboot (such as adding new IP addresses/iptables rules, etc) This also adds to the "risk of downtime" when rebooting infrequently.

Best to plan for some downtime when doing a reboot - or if this is not a desirable option, set your servers up in clusters so that reboots can be done if necessary.

Brent
  • 22,219
  • 19
  • 68
  • 102
8

If you just need security updates and not a completely new kernel, you may be interested in Ksplice - it lets you patch certain kernel updates into a running kernel.

Tim
  • 1,148
  • 1
  • 14
  • 23
3

There's no simple answer to this, some kernel upgrades aren't really security related, and some may fix security issues that don't affect you, while others may affect you.

Best approach imo is to sign up to the relevant security mailing lists like ubuntu's security-announce so that you can see when security patches are coming out, and how they could affect you.

I'd also consider apticron or similar to get the details and changelogs of any other package updates.

theotherreceive
  • 8,235
  • 1
  • 30
  • 44
2

This is a function of the update - if it fixes a priv. escalation that results in root access, then you might want to apply it.

Avery Payne
  • 14,326
  • 1
  • 48
  • 87
2

In the event that you don't reboot, you should ensure that the new kernel isn't the default one to start on boot.

The last thing you want is an untested kernel being used for production after an unplanned restart.

mibus
  • 816
  • 4
  • 5
1

As mibus mentioned, if you install the kernel and don't reboot, make sure it's not the default. You don't know if or in what state your server is going to come back in, so make sure it's tested.

That being said, I think it's good to get in to the habit of rebooting machines on a fairly regular basis when possible. A lot of hardware and software failures will manifest themselves only on a reboot and it's better to find out about those when you're planning a reboot instead of during an unplanned outage.

Kamil Kisiel
  • 11,946
  • 7
  • 46
  • 68
0

Take note that some of the debian kernel updates require (well, highly recommend) that you reboot ASAP after you apply them.

This is the case when the difference is not sufficient enough to warrant a modules directory change, but the modules may differ.

You will be warned by Debian when you install such kernel packages.

MikeyB
  • 38,725
  • 10
  • 102
  • 186