Upgrading Zimbra and your OS at the same time

Question

I've recently had some problems regarding upgrading Zimbra while also upgrading my OS at the same time. To be more specific: I wanted to upgrade Zimbra Open Source Edition from 8.0.5 to 8.6 and at the same time upgrade Ubuntu Server from 12.04 to 14.04. The problem I encountered was that I couldn't get it to work, Zimbra threw a lot of Perl-related exceptions and since the Zimbra forum isn't very communicative I had to reroll to my backup to be up and running after the weekend again. So what is the correct process?

Answer 1

I felt the need to respond after spending two days to get this all working, performing numerous restores due to upgrade issues.

In my situation, it was an update of Ubuntu 12 (Zimbra 8.6) -> Ubuntu 16 (Zimbra 8.8). Sadly, instructions from Zimbra's website (/opt/zimbra/conf/localconfig.xml), and the post from OP resulted in the following LDAP error after performing the task ./install -s on a production version of zimbra:

ldap_url and ldap_master_url cannot be the same on an ldap replica

As a result, the method that worked for me is to perform Labsy's suggestion, which is performed during the do-release-upgrade. To add some in-depth information, for other people that may be confused, you will receive the following message during the Ubuntu OS upgrade process

Updating repository information

Third party sources disabled 

Some third party entries in your sources.list were disabled. You can 
re-enable them after the upgrade with the 'software-properties' tool 
or your package manager. 

At this time, open a SSH session to the server, and edit the file /etc/apt/sources.list.d/zimbra.list and remove the # from the beginning of the lines, so that the zimbra packages will be updated as part of the upgrade process.

Follow the OS upgrade normally, and agree to all default messages. Once the OS upgrade is complete, you can reboot the server. You will know that all has gone well, if the Zimbra services start up during the boot process.

This way you don't need to run ./install -s on the new OS at all.

For others who have come here due to the following message, ldap_url and ldap_master_url cannot be the same on an ldap replica, the reason why you receive the error, is because the ./install -s blows away the LDAP configuration stored in /opt/zimbra/conf/localconfig.xml. If you do not have a backup of this file, you will have to restore from backup/snapshot and start the upgrade process from the beginning.

Answer 2

I found a guide on the Zimbra site that was ok but was lacking some parts (https://wiki.zimbra.com/wiki/Ubuntu_Upgrades). I figured it out eventually but I felt the need to write this down for future reference for me and others. In my case I was using a single server install with Zimbra Open Source Edition, some of this might not apply to other instances.

General stuff

1. Use a virtual Machine and LVM Volumes

Best practice in general is to install Zimbra on a virtual machine and use LVM or another technique that supports snapshots. This way you can create snapshots and in case anything goes wrong you just roll back to your snapshot.

Snapshots can not only be used for having an additional and easy to use fallback point in case an update/upgrade goes wrong but also provide for easier backups:

1. stop the zimbra service
2. create snapshot-volume
3. mount snapshot on the host
4. backup the data
5. unmount snapshot
6. delete Snapshot

Note: Snapshots should be used in addition to backups, not instead of them.

2. Backups

Have a complete backup of your whole /opt/zimbra folder and ALWAYS keep a copy of the Zimbra installer for the version and OS you use. In case your server crashes you can restore your whole Zimbra installation with the folder and the installer.

3. Keep your installation up to date

You don't have to used the very recent version at all times (though it is a good idea) but at least you should make sure that your OS and your Zimbra installation are within their support lifetime.

4. Use screen

In case you administrate your server via SSH, make sure you use screen for all upgrades. In case your connection gets interrupted it doesn't ruin your day (if the terminal session closes it terminates all running commands). With screen you an just resume with interrupted session with screen -r.

Upgrading

Here comes the tricky part. In my case I screwed up and started with upgrading Ubuntu 12.04 to Ubuntu 14.04. Then I tried to install Zimbra 8.6 over my current 8.0.5 and toasted my installation because I became cocky and ignored version checking.

The trick is that you have to install the exact same version of Zimbra on your upgraded OS than you had on your old one. In my case there wasn't even an installer for Ubuntu 14.04 so basically I could only roll back to my Snapshot and start again.

In this example I use my configuration. Again: I wanted to upgrade from Ubuntu 12.04 to 14.04 and Zimbra 8.0.5 to 8.6.

1. Make sure to use the same Zimbra version for both your old and your new OS

In my case, there was no installer for Zimbra 8.0.5 on Ubuntu 14.04, but there was a 8.6 installer for Ubuntu 12.04. It doesn't matter if you upgrade Zimbra or your OS first, but it does matter that you install the same Zimbra version on both your old and your new OS. You can upgrade after you installed your old Zimbra version on your new OS but you CAN NOT upgrade your OS and Zimbra at the same time.

2. Upgrade to Zimbra 8.6 on Ubuntu 12.04

Just download the 8.6 archive, unpack it and install. You don't have to stop zimbra first, the installer will do so automatically.

screen
wget https://files.zimbra.com/downloads/8.6.0_GA/zcs-8.6.0_GA_1153.UBUNTU12_64.20141215195814.tgz
tar xzf zcs-8.6.0_GA_1153.UBUNTU12_64.20141215195814.tgz
cd zcs-8.6.0_GA_1153.UBUNTU12_64.20141215195814
sudo ./install.sh

Follow the instructions. You don't have to to the integrity check if you are asked, but it is a pretty good idea to do so. This might take a while but makes sure your database is intact. After the installation verify that everything is working fine.

3. Upgrade Ubuntu from 12.04 to 14.04

Stop zimbra before you start the upgrade and make sure that all services have been stopped.

su - zimbra
zmcontrol stop
zmcontrol status
exit

Update and Upgrade and then start your release upgrade:

sudo apt-get update
sudo apt-get ugprade
sudo do-release-upgrade

During the upgrade, Ubuntu will ask you multiple times if you want to keep your current configuration. ALWAYS ALWAYS ALWAYS DO THAT aka select Default answer (N)

Configuration file `/etc/security/limits.conf’
    ==> Modified (by you or by a script) since installation.
    ==> Package distributor has shipped an updated version.
    What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer’s version
    N or O  : keep your currently-installed version
    D     : show the differences between the versions
    Z     : start a shell to examine the situation
    The default action is to keep your current version.
    *** limits.conf (Y/I/N/O/D/Z) [default=N] ? 

The same applies when it comes to deleting old packages, just keep them for now. After the Upgrade is done, reboot.

4. After the Ubuntu Upgrade

Zimbra won't work now, don't panic. This is due to the fact that you have Zimbra 8.6 for Ubuntu 12.04 which doesn't work with Ubuntu 14.04. If you didn't do it by now, download the same Zimbra version (in this case 8.6) you had on your old system but this time for the new version:

wget https://files.zimbra.com/downloads/8.6.0_GA/zcs-8.6.0_GA_1153.UBUNTU14_64.20141215151116.tgz
tar xzf zcs-8.6.0_GA_1153.UBUNTU14_64.20141215151116.tgz

5. Install Zimbra 8.6 for Ubuntu 14.04

You need to install all dependencies for your Zimbra installation, in my case it was libperl5.18, but if you don't already have it installed it will ask for unzip etc. Don't worry, the installer will inform you about what is missing, just follow the next steps and if the installer cancels due to missing dependencies just install them and continue. This time you have to run the installer with the -s option and SKIP the integrity check.

The -s option stands for software; it means the installer doesn't upgrade your data, it just installs the new software. You need the new software to upgrade your existing data (mail storage, settings, etc.), if you leave it out the installer will cancel with an error. Before you proceed, make sure again you use the right installer for your version. You see the version inside the folder name: zcs-8.6.0_GA_1153.UBUNTU14_64.20141215151116.

cd zcs-8.6.0_GA_1153.UBUNTU14_64.20141215151116
sudo ./install.sh -s

Important: Skip the integrity check, you can do it later manually after the data has been upgraded.

Do you want to verify message store database integrity? [N]

Proceed and answer with Y if you are asked if you want to upgrade. After the installation is done, run the same installer again, this time without the -s option:

sudo install.sh

Note: You can run the integrity check now or skip it and do it later.

Follow the instructions like before. You should now have a working Zimbra 8.6 on Ubuntu 14.04.

In short

This doesn't work

Zimbra 8.0.5 on Ubuntu 12.04 --> Upgrade Ubuntu to 14.04 and install Zimbra 8.6 on Ubuntu 14.04

This works

Zimbra 8.0.5 on Ubuntu 12.04 --> Zimbra 8.6_for_12_04 on Ubuntu 12.04 --> Upgrade to Ubuntu 14.04 --> Install Zimbra 8.6_for_14_04 on Ubuntu 14.04

Answer 3

Step 3. Upgrade Ubuntu from 12.04 to 14.04

Here it is important to PAUSE when upgrade process tells you about removing non-compatible repositories! Wait before you continue, open another SSH session to the server and edit the file:

/etc/apt/sources.list.d/zimbra.list

Remove hashes # from 2 or 3 lines there, which were added by do-release-upgrade.

Save the file and get back to original SSH session, where now you may continue with upgrade process. Doing so, upgrade WON'T REMOVE, BUT RATHER UPGRADE all Zimbra packages.