How to retrieve old e-mails that were kept in Roundcube's inbox from a backup?

Question

I've got Ubuntu 16.04 (Desktop Ed.) For my e-mail I run SMTP server (Postfix/Dovecot/Roundcube). Once in awhile I do a bare metal backup of the whole system with Clonezilla (plus a tarball file backup too just in case). Since it takes rather long time, I often just do a backup of my www directory structure plus databases that I backup from within PHPMyAdmin. Including roundcube.sql file too. "Roundcube" folder proper resides in one of my sites' (I've got a few of them on one physical machine) public_html directories. ALSO... and that's probably a crucial thing here... I have a content of "mail" folder which itself a sub-directory of another folder named after a user in the system (that I use as a login in webmail interface of RC). I think I created it manually when I was installing Roundcube a few months ago. In that "mail" folder I have ".imap" subfolder too with "INBOX", "INBOX.Sent", "Trash" and other folders too. So I think that basically I've got it all as a backup. That's what I thought... So when my HDD died a few days ago, I easily re-created everything from a Clonezilla image made a little over one month ago. BUT... When I go to RC's Inbox now... I only see the e-mails that were there back then (i.e., a month ago, and not the recent ones!) Import of roundcube.sql doesn't help. I still have those recent emails missing... I want to somehow retrieve them or at least to have access to my recent emails (the ones that were in RC's Inbox AFTER the Clonezilla backup had been made).

Answer 1

The Message Store

Roundcube is not an email server, but an email client a.k.a. mail user agent (MUA); in the Internet Mail Architecture (RFC 5598) it's similar to Outlook and Thunderbird, although it's a webmail written in PHP, runs on a web server and is used via web browser.

The messages are stored in an IMAP server, a Mail Delivery Agent (MDA) including a Message Store (MS). While a MUA could employ a MS (which was typical for POP3), with IMAP the messages are usually stored on the remove server. The messages can be (often partially) cached locally, but caching is less common with webmails. Outgoing messages are sent using an SMTP server (Mail Submission Agent MSA / Message Transfer Agent MTA).

With Roundcube, both IMAP & SMTP servers are configured in config.inc.php:

// The IMAP host chosen to perform the log-in.
// Leave blank to show a textbox at login, give a list of hosts
// to display a pulldown menu or set one host as string.
$config['default_host'] = 'localhost';

// SMTP server host (for sending mails).
$config['smtp_server'] = 'localhost';

The information on the MS accessed using IMAP are in those files you have found in your backups i.e. the folders INBOX, INBOX.Sent, Trash etc. If you haven't got the messages on these files, you don't have backups of them.

Roundcube's database

From the Roundcube Webmail initial database structure in mysql.initial.sql you can deduce that the database contains details that aren't provided by or saved on the IMAP server: session information, users with their settings, contacts, identities with signatures, dictionaries, searches, caches etc.

Roundcube does have an option for message cache (table cache_messages), but it's disabled by default in defaults.inc.php and has some prerequisites for the IMAP server:

// Enables messages cache. Only 'db' cache is supported.
// This requires an IMAP server that supports QRESYNC and CONDSTORE
// extensions (RFC7162). See synchronize() in program/lib/Roundcube/rcube_imap_cache.php
// for further info, or if you experience syncing problems.
$config['messages_cache'] = false;

This is a well-reasoned choice: the default for IMAP server being localhost the messages aren't retrieved from a remote server that could cause latency/unavailability problems. Unless you know you have configured the message cache and how the caching works, it's unlikely you have anything there.

Pro tip of the day: automate your backups.