2

I'm going to migrate IMAP mailboxes from Exchange 2013 to Cyrus with over 10k users. I plan to use imapsync as the tool for this. 

[root@centos65]# imapsync --version
1.644

I'm testing with 5 users and total 134 mails ( most are plaintext + one 9MB attachment ).

First, I tried with below command:

while read user pwd1 pwd2
do
    /usr/bin/imapsync \
    --host1 exchange_svr \
    --password1 "$pwd1" --user1 $user@domain \
    --authmech1 "PLAIN" \
    --host2 cyrus_svr --user2  $user@domain \
    --ssl1 --subscribeall \
    --authmech2 "PLAIN" \
    --password2 "$pwd2"
done < input

Finish time is 1min 57sec.

Using some options in hope to speed up:

  ...
  /usr/bin/imapsync \
--nofoldersizes --nofoldersizesatend --buffersize 49152000 --usecache --idatefromheader \ ## <-- add these options
--host1 exchange_svr \
  ...

Finish time is 1min 49sec, not much faster.

My input file:

user1@example.com   pwd1   pwd2
user2@example.com   pwd1   pwd2
user3@example.com   pwd1   pwd2
user4@example.com   pwd1   pwd2
user5@example.com   pwd1   pwd2

I've tested running multiple scripts, by which each script will deal with some folders (using --exclude or --include) but not really faster ( in fact it took over 2mins ).

Do you guys have any best practices to work with this tool with such many users/mailboxes? Any suggestion for me to make it better?

cuongnv23
  • 230
  • 3
  • 9
  • Yes, it is not so fast. To run it faster run in from one of these servers, so at least one IMAP connection will be local. It also could segfault sometimes. Thanks it is sync tool, it is idempotent, you could simply run it several times with same parameters and it will continue its work, so I write script with commands to transfer all mailboxes and run it until no mail is transferred. – Nikita Kipriyanov Dec 11 '15 at 09:02
  • Yes, actually I ran it on my Cyrus server. As you see, I have a loop to deal with it. BTW, did you work with `imapsync`? How many users/mailboxes and how long to finish? – cuongnv23 Dec 11 '15 at 10:28

1 Answers1

1

There is a FAQ about massive migration in the imapsync documentation: http://imapsync.lamiral.info/FAQ.d/FAQ.Massive.txt

Q. I have to migrate 500k users using 400 TB of disk space. How do I proceed?

Q. How to determine what is the bottleneck in my current imapsync process?

Gilles LAMIRAL
  • 405
  • 2
  • 7
  • Yes, I read your FAQs. But about my additional options, do they really speed up the migration? If I use `admin` acount to authenticate `host1` (using `--authuser1`) does it authenticate once and migrate mailboxes of users via that authentication session? – cuongnv23 Dec 14 '15 at 08:07
  • 2
    --usecache speeds up on next runs of a account but with 500k users you might run out of inodes. --nofoldersizes speed up a little but it removes ETA and final diff. I prefer ETA and final stats than winning few seconds. Just have some measurement, it's my best advice. – Gilles LAMIRAL Dec 14 '15 at 14:31
  • It's him, it's Gilles, the man! – Marco Demaio Oct 12 '21 at 16:45