10

I'm a server guy, specifically a blade guy and know my high-end gear well but I've got a new little challenge that I'm a bit lost on and you may be able to help.

I have a server spec that will do a specific job, I've looked at virtualising it and for dull reasons it's not going to work (not just yet anyway).

The machines have to have;

  • dual core consumer processor with quite low power/heat (I'm thinking one of the newer i3's perhaps?)
  • no more than 4GB of regular desktop memory
  • two PCIe slots - one capable of taking a single-slot x16 GPU (not a 480 or similar, more along the lines of a 9800GT or 240 etc. - about 150W max), the other slot is for a custom low power DSP
  • Built in regular dual-channel sound card
  • a single GigE PXE/iSCSI-bootable NIC
  • I don't need any USB, keyboard, mouse, sound I/O, SATA/PATA or DVD/hard-disk at all.
  • They'll be running either XP or W7 (32-bit).

Now I need to get as many of these into a data centre as possible, I also want them as inexpensive as possible too (given the base specs), something I'm not normally bothered about therefore no expert in. I'll need several hundred or thousands of these machines, and have around an 8KW-per-rack limit (this could go up a little).

I normally use HP blades but even their BL2x220's work out very expensive and don't give me exactly what I need anyway. I looked at SGI/Rackable but they're all server-oriented with Xeons etc.

What are your thoughts? I know this isn't directly server related but it is for professional reasons. Thanks for your help.

Chopper3
  • 100,240
  • 9
  • 106
  • 238

7 Answers7

6

If you are building quite a few of them you might find designing your own case using something like http://www.protocase.com might be the way to go. I believe this is the route BackBlaze took: http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/

Find a consumer or low end server motherboard with your needs and wrap a case around it. If your cards are short enough you might be able to get them into 2U. (I don't think I've seen right angle adapters for PCI-e).

You might also talk to your vendor, if you are a good customer and you need a few of these thing you might be able to get a custom option setup, especially if the vendor decides other customers will want something similar.

mfarver
  • 2,576
  • 13
  • 16
  • 1
    I've seen right angle adapters and risers for PCIe around. Chenbro also does custom case design. This is probably the way to go since he's looking for what is essentially desktop hardware in a tight server form factor; which is a combination I've never seen. – Chris S Sep 01 '10 at 16:13
  • 1
    Some vendors like SiliconMechanics could probably do a manufacture run for you with your custom needs if you have large quantities. I've dealt with SilMech in the past, and highly recommend them. – Urgoll Sep 01 '10 at 16:20
  • I'm seriously considering a custom design with some form or shared power supply - kind of like old telephone slot/racks in a way. – Chopper3 Sep 01 '10 at 16:25
  • You might also be able to do half depth cases and put them both front and back. Have to be careful with cooling. – mfarver Sep 01 '10 at 16:29
  • could use mini-itx with a 12v ps! – tony roth Sep 01 '10 at 20:43
4

How about taking the approach google takes (or at least took at one point). Skip the case, just get a motherboard + needed components and some shelves with some sort of insulation. On a normal 42U 4 post rack i'd imagine you could get 4 or 5 mini-itx type boards on one shelf.

Zypher
  • 36,995
  • 5
  • 52
  • 95
2

If it weren't for the sound card requirement Dell M610x's in an M1000e chassis might be an option. You're going to get the aggregate benefits of more effective power\cooling from a (fairly) modern blade chassis and the M610X will give you two full length PCIe x16 slots that can support the power draw you need. They are full height blades though so not really all that dense but your power budget would kill most denser solutions anyway I think. The main drawbacks are that sound card issue, cost of course as these are decent enough dual socket servers, overall density isn't great at 8 servers in 10u. The 610 platform is overkill for your CPU\RAM requirements but you can configure them with a single low end low power Xeon 5600 and 4GB RAM rather than twin X5690's and 192GB and with those components they are pretty skimpy on power consumption.

You can certainly get PXE\iSCSI boot from the on board Broadcoms, diskless config is fine too. Since your overall power and IO requirements are minimal you can cut costs down on the chassis by opting for 4 PSU's and just a single Gigabit Pass through module rather than a switch.

They do have an internal USB port so it might be possible to meet the sound card requirements with a small USB device. Dimensions are limited to 15.9 mm wide x 57.15 mm long x 7.9 mm and there may be a limitation that this is only for USB storage, I'm not sure.

I'd be surprised if you couldn't configure the stack with <300 Watts per server (and maybe a lot less) under full load even with your PCI cards at full draw so you should be able to get 24 of them in per Rack with that 8kW power budget.

And even though you say you don't need KVM having the centralised chassis management\iDRAC and full plug and play (MAC addresses managed by the chassis) at the server level options for something at this scale is surely a plus.

To be honest though if you are heading into the thousand range then a more bespoke solution might be in order, it's definitely the best way to get a really power efficient solution and one that hits all the requirements without any fudging.

Helvick
  • 19,579
  • 4
  • 37
  • 55
  • Just looked that those Dell blades (HP man myself), they're Xeons and don't have the density I need sorry - quite expensive too really - I can see a custom build coming on to be honest - I'm already talking with Intel and NVidia about getting pure chipsets from them for a specific system design but thought it might be worth asking on here. Thanks. p.s. going for VCAP? I'm going to do the performance track myself. – Chopper3 Sep 01 '10 at 20:32
  • 1
    Yeah thought they might be too server oriented. Actually talking to Intel is a good plan - your use case sounds like the sort of thing their Business Development types would fall over themselves to help you with. As far as VCAP is concerned it's down as one of my end of year personal targets - all I need is for my boss to stop over-scheduling my time between now and then so I can take a decent run at it. :) – Helvick Sep 01 '10 at 20:43
2

The Dell Prevision R5400 is a rackmounted workstation but it uses Xeon processors and ECC RAM, maybe you could make a couple of compromises given the highly specific nature of your requirements you'll struggle to find a product that exactly meets your requirements unless you build them yourself which might be your best solution. Tyan Tank 1u barebones units are well priced and support a wide range of processors.

chunkyb2002
  • 688
  • 3
  • 9
2

Is there a specific reason why you don't use AMD CPUs?

  • 2.5GHz AMD Phenom II X3 has 65W TDP
  • 2.93GHz i3 530 has a 73W TDP

While both have similar performance. Additionally both the CPUs and mainboards are cheaper.

Hubert Kario
  • 6,351
  • 6
  • 33
  • 65
  • No, totally up for AMDs, they'll do the job, are cheaper and cooler - we just happen to have a very strong relationship with Intel but nothing that'll stop me using them - unfortunately the CPU is a small part of the solution I'm looking for, but thank you very much indeed. – Chopper3 Oct 06 '10 at 13:33
  • I just wanted to warn about the "No one ever got fired for buying IBM" fallacy. If you have very close ties/relation with Intel, it may really be a better idea to go along with them. – Hubert Kario Oct 06 '10 at 14:38
2

Are PCI-e slots really needed ? (It's almost impossible to find this with this density)

If not, you should take a look at the Dell Fortuna : http://en.community.dell.com/dell-blogs/direct2dell/b/direct2dell/archive/2009/05/19/dell-launches-quot-fortuna-quot-via-nano-based-server-for-hyperscale-customers.aspx

Kedare
  • 1,766
  • 4
  • 20
  • 36
  • 1
    the op mentions "two PCIe slots - one capable of taking a single-slot x16 GPU (not a 480 or similar, more along the lines of a 9800GT or 240 etc. - about 150W max), the other slot is for a custom low power DSP" - so I'm guessing yes, this custom PCI-e device will need to be plugged in by PCI-e. That is a very cool product from Dell though. – Mark Henderson Dec 10 '10 at 00:48
  • Yep but it's impossible to do that with this density of servers per racks :( – Kedare Dec 10 '10 at 01:03
  • Thanks guys, I do indeed need PCIe and I also know how hard it's proving to deal with this problem. Ultimately we're commissioning a totally custom system from a supplier - it's going to take a while but it's the only way to get it done. Thank you anyway. – Chopper3 Dec 10 '10 at 09:17
1

Rackable/SGI also do customizations (chipset, cpu, etc.), if you're ordering thousands of those then that shouldn't be a problem - i'd talk to them about your specific needs.

pfo
  • 5,630
  • 23
  • 36