How should I set up my home file server?

5

I'd like to set up a home server for three purposes:

  1. Backing up a couple of local computers that are unlikely to ever exceed 500 GB of data each (highest priority).
  2. Serving up files on the LAN - definitely music, probably .iso, and maybe video.
  3. Running extended statistical computations/simulations (lowest priority).

I realize that these goals are not all perfectly aligned; the stats/sim stuff is RAM and CPU intensive, whereas backups need read/write speed and lots of capacity. But I'd really like some advice on how to make this work. Hardware and software suggestions are very welcome. My only requirement is that it has to be Windows, Linux, or Mac (or maybe Solaris, but I'd be most comfortable with the first two). I'm definitely willing to shell out for the hard drives, and I should be able to scrounge everything else - but are there other hardware pieces that I should not scrounge?

Thanks for your suggestions.

EDIT for clarity: I'm definitely not looking help getting set up. Probably the most essential advice I need is on hardware, because it changes so quickly and because I'm unfamiliar with the bottlenecks and wear points that servers of this sort face. I'd also be happy to have any advice regarding software - and, of course, freer is better. Thanks to those of you who have responded so far - I would vote you all up if I weren't so lacking in points.

Matt Parker

Posted 2009-10-08T02:54:02.990

Reputation: 153

4To future readers: these are all really excellent answers. "Best" is marked for the sake of the respondent, but all answers are worth reading. – Matt Parker – 2009-10-09T06:28:23.697

Answers

2

Your usage sounds a lot like mine, so I'll describe my main goals and how they influenced my setup:

  1. Reliable, network-available drive space. Like you I wanted to be able to back files up from multiple computers and serve media files. My primary concern is that the data be as solid as possible, so in the event of a hardware failure, getting to the data is as easy as plugging the drive into another computer.

  2. Cheap hardware. By cheap, I mean "already own". My home servers for the last decade have been whatever workstation hardware I was using before buying my current workstation. Currently, this is a dual PIII-600 with 768Mb RAM, built in 2001 or so. Since putting it into a server role, the only hardware I've purchased for it has been hard drives and drive controllers.

Like you, backing up the family workstations and serving files is my main priority. If the server hardware dies, that's ok -- I can handle a little downtime while I scrounge up replacement hardware, and the replacement hardware can be nearly anything off-the-shelf.

To that end, I don't use any RAID or LVM or DynamicDisk configurations that would complicate moving a disk from one machine to the other. Data is stored on a 1TB drive in a standard ext3 partition, and a second 1TB drive maintains a mirror of the data via rsync. Really important stuff gets backed up to data DVD, and I keep a smaller drive (320GB) as an offline backup of the major stuff.

My home server is limited by the CPU and RAM, but for what it's doing the only practical limitations are network and drive (really bus speed). Graphics/RAM/CPU aren't really a factor for me, but they'll be factors for you if you optimize your hardware for simulations and computational jobs. If this is a headless server, you could even take advantage of a good GPU for your computations.

As far as software goes, I highly recommend Linux. If you're new to Linux, consider Ubuntu or Xubuntu, especially the server form. (Use Xubuntu if you end up with the desktop version; GNOME is a resource hog). On the other hand, if you're very comfortable with Linux, consider a very light Debian install instead -- a typical home server doesn't need the heavyweight desktop GUI stuff, and your usage would benefit from having as little of it as possible.

quack quixote

Posted 2009-10-08T02:54:02.990

Reputation: 37 382

2

There are way, way too many possible answers to the question.

If you're a Windows person and will be backing up only Windows PCs, Windows Home Server has a great utility to back up workstations automatically.

I'm more of a Linux guy, and I would use an inexpensive PC running Debian (the server distro I'm most familiar with) and a terabyte drive or two.

I'm not utterly certain what you're asking for--do you want a detailed "how to set up the server" document, or just general suggestions?

CarlF

Posted 2009-10-08T02:54:02.990

Reputation: 8 576

2

You can pretty much scrounge everything BUT the hard drives - data integrity is the most important.

For the first 2 objectives you are pursuing, you do not need a powerful computer. Essentially an Atom chipset will do just fine. You will want to hook this computer up via wire (Gigabit preferred) to your network though, for network transfer speed is going to be the main bottle-neck here.

With regards to your 3rd objective of running stats comps/sims, well it really depends on how long you can wait for results to come out. If you need it to be fast, then get a faster processor, and more RAM. Be careful though - this objective might slow down your disk I/O operations depending on what kind of comps/sims you are doing.

Finally, just make the box a SAMBA share and use a backup software such as Cobian to schedule regular backups to the box. Run some monitoring tool, and most importantly, have fun!

caliban

Posted 2009-10-08T02:54:02.990

Reputation: 18 979

+1 good points, esp about wiring the network. re: comps/sims vs diskIO, wouldn't (re)nice'ing the comps/sims at runtime take care of that? disk access is likely to be bursty for data access, with regular spurts at backup time. – quack quixote – 2009-10-09T06:29:24.960

2

Freenas is easy to setup, only needs like under 256 megs for the OS, runs a slimmed down FreeBSD with lighthttpd as a webserver. and the install is quick and painless. Can run on ancient machines, like rub 500mhz and only really needs like 128 ram.

Then use this to do network backups of all OSes on your network.

alpha1

Posted 2009-10-08T02:54:02.990

Reputation: 1 638

TimeDrive sounds interesting, thx for the link. – quack quixote – 2009-10-09T06:39:15.797

2

What you should also keep in mind is power consumption. A server that only backs up your computers and does some file-serving won't cost too much power. One that is computing statistical thingamajiggs 24/7 will cost some more, and depending on where you live this may add up to quite a lot of money.

That said I'd go for something like an Atom - it'll work perfectly for your higher prio items, and may even do some calculations. It will be slower, but it won't cost you as much in the long run.

Emthigious

Posted 2009-10-08T02:54:02.990

Reputation: 343

1

I would (and did) set up a Linux server for SMB/CIFS (samba) and AFP (netatalk, if you have Macs).

One strength in using Linux is that it suits a homogenous environment well. I have two Windows systems and a Mac system on my network connected to the file server. It can also be a single-sign on server via LDAP/Kerberos/Samba, yes even to Windows systems.

For hardware, I suggest 2-4G of memory, and separate hard drives for the shared data. Use RAID-1 mirroring, or RAID-5, depending on how much your budget is for hard drives. I suggest hardware RAID as it is transparent to the OS. Since Linux is free vs the cost of a Windows license, you can add another hard drive :-).

I also have my file server set up to be a Time Machine backup system for the Macbook I work on. I manage all the configurations with Chef, so I can rebuild the machine from scratch easily.

jtimberman

Posted 2009-10-08T02:54:02.990

Reputation: 20 109

-1 no RAID. boooooo. especially not hardware. if it breaks you can't fix it without replacing with the same chipset. RAID is not a backup, it's a headache and a disaster waiting to happen all rolled into one. (kidding about the -1. not kidding about RAID. you've been warned.) – quack quixote – 2009-10-09T06:36:12.747

1

I just put together a standard computer in a rackmount case. Quad core, 6GB, Server 2008 (free, which is a long story). Just set up UNC shares and map my workstations' local drives to the drives on the server, i.e.

  • C: System
  • D: \server\d$
  • ...
  • L: \server\l$
  • Z: Local DVD drive

That way my docs are always in the same path (in my case, i:\docs), etc.

tsilb

Posted 2009-10-08T02:54:02.990

Reputation: 2 492

0

If some of your local computers are Macs, I recommend this tutorial on how to get Time Machine working with a linux server. Time Machine is a great way to backup, and by using a linux server you can achieve your other goals, too, if you spend enough time on it.

Karl Bartel

Posted 2009-10-08T02:54:02.990

Reputation: 361