0

I'm planning to move one of my development database servers to its own machine. I'm currently running it on my laptop, which is not very convenient.

  • DB Servel is PostgreSQL 9.0.1,
  • OS is Gentoo.

There is only one development server attached to the db, so there's not many queries. The problem is the amount of data: I need to store a copy of planet.osm, which is over 170 Gb at the moment, and increasing.

I'm looking for a solution that doesn't require me to buy expensive hardware, as this is not a production server anyway. I might even be able to reuse some low-end machines I have, with the only exception being their hard drive.

So my question is, what kind of hard drive should I be looking for?

I am currently using a relatively cheap 1.5 Tb external USB drive with a single btrfs partition on it, but I'm not sure that would be fast enough for a db server to find a needle in a 170+ Gb haystack on it.

What tools can I use to measure hdd performance on Linux?

Note that this question is somewhat related to this one, with the main difference being I'm not building a production server.

Community
  • 1
Attila O.
  • 226
  • 3
  • 12

3 Answers3

1

I would not use a USB drive -- The performance will be suck-tastic. Invest some money in a machine with moderately fast SATA drives. Consider RAID if you think you'll need the redundancy.
You can get a 1U system that's suitable for development for around $1000 or less (think Iron Systems or really low end Dell -- Remember that performance is not paramount in development environments, it just needs to be "good enough").

Another option would be spending $6000-10000 on a VMWare host & virtualizing your whole environment (this would be my suggestion -- It makes life a lot easier & you get the flexibility of being able to spawn copies of your development environment to try out changes or do pre-release testing. Downside here is virtualized performance is almost always worse than a dedicated box, but like I said above performance in Dev isn't always the ultimate goal.

Re: your benchmarking question, this google search is probably a good place to start, but honestly I wouldn't worry about it for a development environment. Benchmarking disk is really most useful when you're comparing systems...

voretaq7
  • 79,345
  • 17
  • 128
  • 213
  • Yep, while the USB drives are handy for a laptop, it seems I'll need to go with SATA drives. I don't plan on virtualizing mu whole environment right now, although that might be an option to consider for the future. Thanks for your help. – Attila O. Oct 22 '10 at 16:12
1

If you want mainly read performance, I'd suggest going the RAID10f2 route with either few inexpensive 1TB drives (Samsung F3s are quite good) or if you really want low access times: WD Velociraptors -- those are 10k rpm disks with SATA ports.

Linux RAID10 dirver can do RAID10 on 2 disks or more, so you can easly scale. While the f2 layout has read performance of RAID0 on the same number of disks.

Hubert Kario
  • 6,351
  • 6
  • 33
  • 65
  • Thank you very much for the hard drive suggestions. I think I'll go with a RAID10, although I'll need to read more on the subject. But these are good advice for starters, thanks. – Attila O. Oct 22 '10 at 16:15
  • You need to remember though that the f2 layout is a bit slower than the n2 (the typical configuration) with write, it's nothing major: about 10-20% slower for streaming writes, random will be even slower, but I don't know the specific numbers. – Hubert Kario Oct 22 '10 at 16:34
1

I would recommend a pair of Samsung 840 SSDs in RAID-10. You will need to tweak Postgresql to get best performance, but for small random reads you should expect a performance boost of something like 15x compared to a hard drive.

Hugh Bothwell
  • 111
  • 1
  • Thanks for reviving this ancient post, but I'm afraid I don't even have this hardware any more :) Good tip on the SSDs though. – Attila O. Feb 22 '14 at 04:00