How to install and manage software on a server?

Question

This is probably a question where the full answer fills entire books; but I'm not a professional IT administrator, so an overview/intro would be helpful already.

Suppose I have to manage a Debian server (virtual server at a hosting provider, with root access) running Apache, Django, some self-written PHP scripts for administration, Nagios/NRPE for status monitoring, SFTP for private file upload. The server should mainly serve static files uploaded by SFTP; Django will be used for two small dynamic sites.

From which sources should I install software like Apache, PHP, Django, NRPE, Nagios plugins? I can think of these possibilities:

• use official Debian packages
• self-compiled from official TGZs, installed in /usr/local
• download and install via git clone from upstream repository, into /opt or into home directory
• install with PIP, Gem, CPAN (where possible)
• use Docker images

Which of these should I use? Are there other useful possibilities that are not listed? What guidelines should I use when evaluating all of these sources? Are there common practices for choosing the best installation method?

Answer 1

To my mind, the correct order of origination is:

1. Using the official package management tool and the official repos
2. Using the official package management tool and unofficial repos, carefully vetted for reliability and stability
3. That's it.

You do not want to get into installing software by hand. It very rapidly becomes an insane maintenance load (you have to check all the project pages, every day, to see if there's some new bug that affects you, and every time there is you have to schedule a rebuild, and roll it out). Installing binaries that you've just pulled from the internet is even crazier; at least with unofficial repos you have your package management tool's signing provisions to rely on: you can't be sure that a randomly-selected set of PHP RPMs came from remi's repos, say, but you can be very sure that all versions you install from those repos came from the same place.

There is always the counter-argument that foo is a special snowflake; that it's just libfoo that your developers desperately need at version 1.4.5.a3_345.21z. And in practice, this is sometimes actually true. But it isn't true anything like as often as the devs make out, and your pushback should be mighty, so that their pain required to get libfoo supported outside the regular channels is comparable to your pain in supporting it.

And even then you should package it, and have a company-internal repository whereby the package is made available.

Answer 2

While MadHatter does an excellent job of answering the original question I'd like to answer the comment

So what about software that is mainly available as a Gem or PyPy package? If I understand you correctly I should try to avoid such software, or use a Debian package (even if it's older)?

along with

Are there other useful possibilities that are not listed? What guidelines should I use when evaluating all of these sources?

from the original question.

First, you should probably use the distribution-provided packages where they are available. This will get you reliable security patches and bug fixes. This applies to gems and python packages also. Many are available packaged by your upstream or some other volunteer project. When you get beyond the available packages use the language-level packager, but have it maintain things in a directory separate from the system binaries and tell your app to add that library directory.

Second, whether you can live with an older package may be a hard decision. Keep in mind that keeping up with the most current software is a lot of work. If you really want to go down that road, read up on continuous-integration systems like jenkins or Travis. When you've got so many moving targets the only way to keep ahead of it is to continually test every new iteration.

Since you say you are using a "virtual server at a hosting provider" it might be good to keep in mind: it could die at any time. You might need to rebuild it on short notice. (That's just life in the cloud.) Or you might decide you want to build more of them if your organization is really successful. This leads to backups and configuration management. Backups are a great idea, but I will let somebody else tell you all about them. I want you to consider configuration management for installing your software.

There are a variety of configuration management systems -- cfengine, puppet, chef, and salt are examples -- but for ease of getting started at a small scale I'd definitely recommend ansible for your situation. You could write shell scripts to install your various types of software, but it will be easier and more succinct to let ansible do it for you.

First I can install some system packages using apt-get:

---
- name: install deb's needed by users
  apt: name={{ item }} state=present
  with_items:
   - git
   - git-svn
   - ansible
   - tmux
   - screen
   - whois
   - traceroute
   - mtr
   - curl
   - wget
   - pcregrep

and then install a Ruby gem:

- name: install vagrant 1.0
  gem: name=vagrant version=1.0 state=present

and a python package:

- name: Install (Bottle) into the specified (virtualenv), using Python 2.7
  pip: name=bottle virtualenv=/my_app/venv virtualenv_command=virtualenv-2.7

You could have installed multiple pips or gems in either of those if you followed the with_items syntax we used with apt. You mention installing a PHP application. Hopefully you're storing this in source control somewhere like github so you can use the git module to install that too. As you continue building your ansible playbooks you end up with an easy way to reliably reinstall your systems without needing to keep lots of obscure details in your head.