Fast and reliable transfer of multigigabyte directory to remote Windows machine?

Question

I need to transfer 100K+ images (~15GB) to a Windows VM running on Azure. I used standard Windows Remote Desktop resource sharing to allow access to my local drive and tried drag-and-drop using File Manager. That transferred fewer than 10% of the files overnight.

I then tar’red up the images into a single file and d-and-d’ed that. That transfer seems to be going a little better but still estimates 6 hours and the upload speed is vastly below my connection speed.

Is there something cleaner than setting up a temporary FTP server?

Answer 1

If you are sure that the upload speed is far below your possible bandwidth, meaning that you have a very fast up bandwidth, you may use Fast Data Transfer, a high-speed data loader for Azure, described as :

The tool works by maximizing utilization of the network link. It efficiently uses all available bandwidth, even over long-distance links. On a 10 Gbps link, it reaches around 4 TB per hour, which makes it about 3 to 10 times faster than competing tools we’ve tested. On slower links, Fast Data Transfer typically achieves over 90% of the link’s theoretical maximum, while other tools may achieve substantially less.

The above article provides a link to a PDF file with installation and usage instructions.