What is the most efficient way to encrypt and upload many TBs worth of information to a cloud service?

I am wondering if anyone here had any suggestions about way to upload an entire external hard drive to a cloud service. I currently have an external hard drive with 4 TB's that I would like to upload to something like Google Drive. Before doing so, I'd like to encrypt it using something like 7zip before uploading it. I've naively tried to hookup another 5 TB drive as the destination for the encrypted file, but it is taking much much longer than I thought. Are there simpler ways one may do this? Thanks!

user321627

Posted 2017-07-19T03:17:09.130

Reputation: 191

Depends how much you're willing to spend and how you define efficient. For example Backblaze has a service where they ship you a drive array that you upload your data on, then ship back. AFAIK it's only available if you're subscribed to their B2 service. I believe AWS offers this as well...

– wysiwyg – 2017-07-19T03:51:31.897

Yes, crashplan, has a similar option. I upload about 2x that, and it took 1.5 years to complete at 2mb/s. The data was compressed so it would be even more otherwise. They also offer encryption options. If speed is your goal just get 10x more upload speed, sure it cost more, but it will be way faster. – cybernard – 2017-07-19T03:55:12.333

@cybernard If your ISP is a telco, and you're lucky enough that they're bringing (or have brought) fiber optic right to the door, then cost of fast upload speed won't be an issue once the fiber optic is in. Upload speeds on fiber are as fast as download speeds for no extra cost. – RobH – 2017-07-21T19:02:37.017

@RobH I have a gigabit connection, in that case, is there a certain way of doing things? – user321627 – 2017-07-23T02:52:50.297

@user321627 We don't have fiber in yet, so I've never backed up to the cloud. There are several web sites that let you check your broadband speed, so you should use one to see what your upload speeds are like. (Just google 'internet speed test'.) Unless you're on a fiber optic line, your upload speed will usually be about an order of magnitude or two less than the download speed. If that's the case for you, then there's really not much that you can do about it with your current ISP. – RobH – 2017-07-24T16:10:24.393

1Rather than encrypt the entire lot into one 4 TB file, it might be better to encrypt the data into many smaller archives. If you ever need to get the data back, there are many opportunities for a 4 TB download to break. Also, if you know the required data is in a particular archive then you don't need to download 4 TB to get, say, 1 GB. – Andrew Morton – 2017-07-24T17:29:29.327

Answers

At the enterprise level, Veeam, Microsoft DPM and other EMC products can be an option but it totally depends upon the backup requirements. you can use any backup software integrated with cloud storage provider.

I tried CloudBerry Cross-Platform Cloud Backup which provides a simple GUI to manage backup and restores and cloud storage account comes bundled with the software. It also provides enterprise backup features like job scheduling, CLI, compression and encryption.

mukulsingh50

Posted 2017-07-19T03:17:09.130

Reputation: 1

The whole scenario depends on your bandwidth. Depending on your ISP capabilities you can evaluate how much time will you spend for this task. Nowadays you should evaluate how much time and money will you spend by uploading such a big amounts of data somewhere offsite or to the cloud.

I agree with the previous answer about many backup software tools that allow to seed the data and then automatically synchronise it with your cloud account.

Also, you should consider where you'd like to store your data.

For instance, AWS offers the service called AWS Import/Export Snowball. With this service it's easy to ship the Terabytes of data to your bucket without paying extra money to your internet service provider.

On the other hand, if you want to land your data somewhere other than AWS - you can use any backup tools. Some of the mentioned above also have archive mode. This option allows you to utilise less API calls for seeding a lot of files. I see that it's also a requirements for you.

Alex

Posted 2017-07-19T03:17:09.130

Reputation: 101

If you have good speed, you need a bit of patience to upload all the data. It will take time in any case, even if you encrypt it as the amount is big enough. As for the encryption, any third-party backup tool can do it for you e.g. Acronis, Cloudberry, Veeam etc. offer such an option. Some of them give even a client-oriented encryption for a person to feel safer and more secure. A third-pary tool is an option in your case to do with things quicker. In most cases you don't pay for them and they work with all modern cloud storages.

Mary17

Posted 2017-07-19T03:17:09.130

Reputation: 1