What is the main bottleneck for a personal offsite backup server

I want to host a personal cloud server on my network and am currently at a crossroads with hardware requirements for the server. My first thought was spend big bucks on a high performance machine with lots of data storage and memory but then I got to thinking that in terms of pulling data externally from the server, a lower powered machine would perform just as well because the internet connection would be the main bottlneck - never fully able to saturate the resources of either the lower resource server or the higher.

For instance, a rasberry pi with a 2 TB usb hard drive would perform just as well as a server packed with dual xeon CPU's 16 gigs of ram and 2 TB worth of 10000k drives.

Is this true?

Locally, I see there would be an advantage to have the beefy server but remotely I am not so sure.

Scandalist

Posted 2014-03-10T02:58:37.413

Reputation: 2 767

2If it's on your network, how is it 'cloud'? How is it 'offsite'? How can anyone tell you if your internet connection is the bottleneck if you don't say how fast it is. You could have Google Gigabit Fibre for all we know. What would you be pulling from the server - fast enough to stream DivX? Blu Ray? Copying everything in a month? Copying everything as fast as possible? Syncing but not copying, as fast as possible? How many 10k RPM drives? The rPi+USB would be much much much worse for random reads. – TessellatingHeckler – 2014-03-10T03:03:47.393

Wordage is a little wierd. Its a personal cloud server because I own it and would access it remotely via my web browser in my travels. In this context it would be offsite for me. Currently I have regular broadband from Comcast, 30Mb down and 6Mb up. The server would not be streaming but only syncing with my devices wherever I happen to be – Scandalist – 2014-03-10T03:07:04.470

6Mbits/sec = 0.75MByte/sec that would be a bottleneck for raw transfer out from your house to your devices; USB drive read rate is ~25MByte/sec. But it might also matter what you are syncing - a single hard disk gets to ~100 IO requests/second; if you rsync thousands of small files, rPi processor/memory limits and disk IO limits might slow you down before any files are transferred. Consumer USB drives aren't built for 24/7 use, and there's no server class hardware or RAID disks in the rPi setup, so I wouldn't want to rely on it; an older or low-end server would do for less than 'big bucks'. – TessellatingHeckler – 2014-03-10T03:27:39.970

Answers

It depends a little on how much data you're backing up, and a little on what risks you are mitigating. Other factors include how you are doing the backup (e.g. what software / protocols / technologies), and how frequently.

Bandwidth is almost certainly going to be a bottleneck, so put some thought / money into that one. Often people mitigate that partly through scheduling (you can move a lot of data overnight).

If you're going to use encryption or depend greatly on compression, then CPU will start to matter a bit. If you're going to use CPU or RAM intensive technologies (e.g. rsync, data deduplication or the like), then it makes sense to consider trading off some other feature for more processing bang. You rarely have to worry much about this, but it might not make sense to go with the lowest-end CPU.

For a backup server, it rarely makes sense to buy really really fast disks. You may need faster disks if you are writing to a tape drive that has a high minimum transfer rate (yes, some drives do), but for a personal server you can probably get away with a cheaper slower tape drive and slower disks. But make sure the disks are reliable, and RAID them for redundancy (RAID1, 5, or 6). Bulk writes mean RAID5 doesn't hurt as much as it normally does.

More disks typically means more storage and more speed, which you can use to store backups, but also to manipulate backups and to perform restores. Buy at least twice the storage you plan to need to hold backups, possibly four or more times that much.

Hope that helps.

Slartibartfast

Posted 2014-03-10T02:58:37.413

Reputation: 6 899

Thanks. I'm actually thinking of using bit torrent sync to establish a remote backup solution for one of my clients. Currently, they have about 60 gigs of data tops between the 3 of them and only work with a few files on a daily basis. I'd like to go low power so I was thinking the rasberry pi with an external hard drive attached ought to be adequate. At most maybe 50 MB of data would be synced each day. – Scandalist – 2014-03-10T07:57:33.140

Keep in mind that synchronization is nice (you don't lose data if a disk dies), but it doesn't help you with corruption that you don't see until after the sync happens. – Slartibartfast – 2014-03-11T03:42:52.737

If you plan on storing alot of data safely I'd look at a NAS device that supports RAID.

I bought a QNAP TS-419P and very happy with it's performance. It's way more than a storage device and works as a perfect "home cloud" out of the box. I felt they were a bit pricey when I bought it but it's done nothing but please ever since I bought it.

http://www.qnap.com

micwallace

Posted 2014-03-10T02:58:37.413

Reputation: 182