6

I've to do a migration of two servers with large SAN attachments to our new VMWare environment.

EDIT: I have to supply some additional intelligence as I have good answers regarding VMWare solution.

Ok so, I can't attach a previous EMC LUN on the New system due to some technologies limitations on the server.

I can't use VMWare Converter to clone the missing volumes on my new VM as VMWare Converter can't see EMC PowerPath Pseudo-devices and that the previous admin used those Pseudo-device to built LVM2 and/or ASM volumes on top of.

Those two physicals servers are attached to an old EMC² CX-340 SAN and handle 5TB of data.

Those 5TB of data are small PDF and I need to transfer them to the new machine through our 1Gbit/s LAN.

I've tried using rsync, but it's really to slow and have a strong impact on the RAM and CPU performance.

I've try using NC with TAR but the transfer rate is quite slow as I've an average throughput of about 50MB/s on a 1Gbit/s link with barely zero traffic.

Could you give me some advice or return of experience with this kind of migration and how you manage to have it finished correctly within reasonable amount of time?

Dr I
  • 943
  • 16
  • 33
  • 1
    If you indeed get an effective 50 Mega **Byte** per second (multiply by 8 to get Mega **bit** /s) that is already 40-50% of your maximum wirespeed so not too bad... Since PDF's are already compressed you can safely omit any compression options. With many files you also can run into some file-system overhead and simply copying the whole SAN LUN with `dd` might be more speedy. – HBruijn Apr 27 '15 at 09:59
  • No, indeed it was a Typo mistake, I've an average of 50Mbits/s on a 1Gbits/s link and so about 5MBytes/s transfer rate, which is really slow as I already achieved 720Mbits/s transfer on this link with another machine but with large files of around 300GBytes. – Dr I Apr 27 '15 at 10:02

4 Answers4

10

If you really need a quick way to transfer files, and both systems are Linux-based, you can try UDR.

This is really a form of rsync-over-UDP (using the open-source UDT framework) and is particularly handy for moving large numbers of files or transferring over high-bandwidth or high-latency links. In addition, encryption is disabled by default, so the RAM/CPU hit is lower than traditional rsync. SSH is not involved either.

I can easily get wire-speed transfers over 1Gbps with 10-million small TIFF files in a directory tree.

Your syntax will be slightly modified from rsync. All rsync flags need to appear before the source/destination specification:

udr rsync -avP --stats --delete /data/ server2:/data/

Easy to build... You'll need g++ and openssl-devel:

git clone https://github.com/LabAdvComp/UDR.git
cd UDR/
make
cp src/udr /usr/local/bin/

Do that on the source and destination.

See: Possibility of WAN Optimization for SSH traffic

Community
  • 1
ewwhite
  • 194,921
  • 91
  • 434
  • 799
  • Thanks a lot for your deep insight on this topic, but what is the difference between UDR-Rsync and a Netcat/Rsync or Netcat/TAR in UDP transfer? As I try on my relatively small test server, with NC/TAR I don't have encryption or other overhead (Except for archives but its not significant) and achieve the test on a 1TB set of data with around 45 Millions of PDF files, my average bandwith is... well... around 2 or 5 MBytes/s on a 1Gbps link. – Dr I Apr 27 '15 at 11:46
  • @DrI UDP is the real difference. Plus, UDT/UDR is an optimization tool purpose-built for data transfer acceleration. Netcat is _just_ netcat. – ewwhite Apr 27 '15 at 11:52
  • Yeah that what I read on your optimization link, the "magic" is not coming from the UDP stack but rather from the UDT library which is something similar to the diagram optimizer provided by silverpeak on its WAN Optimizer solution. I'll give it a try after the sFTP test. – Dr I Apr 27 '15 at 11:57
  • 1
    SFTP will be slow for the same reasons rsync+SSH will be. – ewwhite Apr 27 '15 at 12:00
  • @ewwhite, Yep, you're right, It's exactly the same things as the pipe use SSH tunnel and checksum. Just tested it, I've got the exact same 5.5MBytes/s throughput. – Dr I Apr 27 '15 at 13:27
  • Hum... I just think of using a combination of Netcat on UDP mode + DD as a block transfer is always faster than a file transfer. What do you think about that solution? – Dr I Apr 27 '15 at 13:31
  • @DrI I gave you my recommended solution in the answer above. – ewwhite Apr 27 '15 at 13:32
  • I just want to test the best options with default commands before trying to use a faster and better tool but possibly rejected one by the company policies. I swear that I'll test your tool (as it really seems to be the tool that I need) as soon as all my native options will be out of the performance window. – Dr I Apr 27 '15 at 13:35
  • @ewwhite Ooook, so I did test different scenario, using NC, DD, rSync, SFTP, VMWare, UDT/UDR, I suspect I've got some sort of HW bottleneck somewhere as I'm always using about 5MBytes/s bandwith any technics that I should use. I'll talk to our storage area manager. – Dr I Apr 28 '15 at 09:37
  • Can you test between the two servers with [`iperf`](https://iperf.fr) and get back to us? – ewwhite May 02 '15 at 12:18
3

If used in daemon mode without encryption, rsync can efficiently transfer large amount of small files. Give it another try using it in daemon mode.

shodanshok
  • 44,038
  • 6
  • 98
  • 162
1

Have you not thought of exposing the SAN LUNs directly to the new VMs - this generally works just fine and can be faster than copying the files into a VMDK - though it can 'lock' the VMs onto their initial host. But you could use this to get things going then migrate the files into a VMDK at your own pace - with rsync - and later cut the link to the original LUNs.

Chopper3
  • 100,240
  • 9
  • 106
  • 238
  • Oups, I forgot to mention that I can't attach the old SAN LUNs to the new VMWare Infrastructure. But yeah, originally I would have done it just like that if I had the link. – Dr I Apr 27 '15 at 11:41
1

If the destination VMs aren't yet built, you might try using the free VMware Converter to copy the data over.

In fact, even if they are built, you could clone the disks to a dummy VM then attach them to existing VM once the clone is done.

In any event, the converter uses two methods to clone files from source to destination, the full details of which can be found here.

If the destination disks are configured to be smaller than the source, it will clone individual files into the new VM.

However, if the destination disks are setup to be equal or larger, it clones blocks. This would make the amount of files on disk pretty much irrelevant and it should run relatively quickly.

I doubt you'll fill a 1Gbps pipe, but you should get more than 50Mbps.

Just remember that you're still looking to move 5TB, so it's going to take some time.

GregL
  • 9,030
  • 2
  • 24
  • 35
  • thank you very much for your answer, I've edited my question to improve the informations regarding my situation as I didn't provided enough insight earlier. Anyway, long story short, VMWare Converter can't clone/find/see all my attached disks as the previous admin used PowerPath. – Dr I Apr 27 '15 at 12:40
  • In theory, if the running OS can read the disks, so can the Converter Agent. Assuming of course you use hot clone mode, instead of cold clonning. – GregL Apr 27 '15 at 12:42
  • I use the hot clone mode, my VMC is not able to see the pseudo devices, I've found numbers of people facing the same issue with powerpath pseudo-devices which can't be cloned as /dev/emcpowerXY device is not detected by VMC regarding some obscure reason. But thanks a lot for the in deep KB post about VMC anyway. – Dr I Apr 27 '15 at 12:44
  • 1
    Hm, well then. I don't know what to say! – GregL Apr 27 '15 at 12:45