9

I work for a company of about 50 people. Our two servers (identical) are custom built with these specs:

Xeon E3-1270 V3
Intel Entry Server Board
32 GB DDR3 ECC
2x 256 GB SSD RAID1 (for System)
4x 1TB SSD RAID10 (for Hyper-V VMs)
Windows Server 2012 R2 as Host and VMs

Each server hosts two VMs (2x AD + file sharing + roaming profiles, 1x SQL server for testing, 1x other (not important).

We perform daily backups using the built in Windows Backup on iSCSI targets hosted on a QNAP NAS (2x 2TB RAID1).

The servers don't have a lot of load and we never had any problems. Most of our data is stored in the cloud (VS Online, SharePoint).

But I wonder if it's sensible continuing this setup or if it's better to switch to professional server hardware, namely one big machine.

So what are the pitfalls and what should I do about them?

MadHatter
  • 78,442
  • 20
  • 178
  • 229
Martin Walter
  • 207
  • 1
  • 7
  • 2
    "*But I wonder if it's sensible continuing this setup or if it's better to switch to professional server hardware, namely one big machine.*" These are two rather different questions, if you are also looking for a comprehensive answer to the latter question then it may be worth posing separately. Aside from that, can you clarify your role in the organisation? I'm assuming you're in charge of the infrastructure but if not you obviously don't need to do anything. – Lilienthal Sep 15 '16 at 10:44
  • 1
    I can think of several different answers here so identifying a core question (is this setup ok? is this hardware ok? should we consolidate services? ...) may help you get more useful answers. – Lilienthal Sep 15 '16 at 10:44
  • @Lilienthal I'm the main admin of the company. Yes, I want to know if the hardware and setup are ok. – Martin Walter Sep 15 '16 at 12:07
  • Whether the setup is ok is a pretty broad topic as it involves a lot of factors: Do you have an offline and/or offsite backup? Do you have a recovery strategy? Have you tested said recovery strategy? What downtime can you stomach? What's your budget and expected level of service? What's your update and upgrade process like? Are your systems and credentials documented? Have you considered bus factor? ... That's a pretty different question from "when does it make sense to move away from consumer hardware?" – Lilienthal Sep 15 '16 at 12:20
  • @Lilienthal I know it's a broad topic and there are many things to consider. I mostly want to know if I'm headed completely in the wrong direction or if it's ok to run our company IT that way in general. So more a basic assessment (It's an ok setup/I would probably do something/Are you insane) – Martin Walter Sep 15 '16 at 12:28
  • Understandable and there's nothing wrong with that. I was looking at it more from the perspective of having/creating multiple, more general questions on these topics as I believe each deserves its own question. Your question is more focused on an assessment of your specific environment. But I'm not a regular here so I don't know how this community handles that distinction. – Lilienthal Sep 15 '16 at 13:53
  • @Lilienthal Consumer grade? – Ryan Babchishin Sep 15 '16 at 15:56
  • @RyanBabchishin Typically business run on "enterprise" class hardware: servers in a rack configuration, Cisco routers and switches, advanced (expensive) hard-drives and the like. By contrast your laptop or home desktop is consumer grade, though there are a variety of terms used for both types of hardware. The distinction is usually that enterprise hardware is meant to be more durable, to warn about possible errors or recover from them more easily or to provide advanced features that a home computer network doesn't need. – Lilienthal Sep 15 '16 at 17:06
  • This kind of setup is considerably more expensive and system administrators sometimes use consumer hardware in their systems to reduce costs at the expense of reliability or uptime, among other things. // And now I'm realising that you may know what I mean but are asking why I mentioned it when OP seems to be using enterprise grade in his system list. I glossed over his list and assumed from "*if it's better to switch to professional server hardware*" that he wasn't currently using server grade components. – Lilienthal Sep 15 '16 at 17:08
  • 1
    @Lilienthal Lol. I love how you caught yourself sort of mid paragraph. Yeah that's what I meant. – Ryan Babchishin Sep 15 '16 at 20:50
  • I erased my answer, and did a v-t-c for too broad, as I realized that for your question a consultant should do the job. As we have no idea of your switch backbone, ups, etc... Everyone on the internet could give you advice, but in the end only your budget versus what you should upgrade is important, and for that an real analyze from someone familiar is important. – yagmoth555 Sep 16 '16 at 15:30

5 Answers5

11

Sounds like you're already using decent hardware. What's wrong with it? It's not too old right? Keep your stuff in warranty or close, if you don't want to feel too concerned (not everyone shares my opinion on this).

If you have stuff setup to be redundant and good backups, you're doing pretty good. One server = single point of failure, no matter how good it is, that would make me uncomfortable. There's a lot you can do on a budget by making smart decisions around how stuff is implemented software/hardware/infrastructure/support wise.

If you don't have precautions/things in place, maybe you should be worried. If one system dies, are it's services are gone? How will that effect business? How fast can you recover?

Pitfalls? Depends. You didn't provide too much info. Cheap drives can fail or be slow. Cheap cases can overheat. Cheap fans can fail. Cheap SATA/SAS/RAID controllers can screw up or not perform as expected. Cheap power supplies can die or if not redundant, leave you without power. Motherboards can do wonky things. Systems without remote consoles (ILO, etc...) can be a pain to manage. Cheap network cards can have cheap drivers or screw up. Lots of little unforeseen issues can occur. On the other hand, you can get cheap as hell entry level stuff that performs amazingly. And more expensive stuff can be wonky sometimes too.

I've seen it all, in decent* server grade, lower end server, workstation and consumer grade equipment. Higher end stuff seems to do better in the long run (way past warranty). But if you can't afford it? Or you can only afford one server and can't implement proper redundancy?

There's nothing essentially wrong with dual servers running with Xeon's, ECC memory and RAID. Unless you have a problem with it.

Ryan Babchishin
  • 6,160
  • 2
  • 16
  • 36
  • It may be helpful to first define the goals (things you absolutely need, things that are nice to have and things that don't bother you much) and the budget and then buy the best quality that fulfills those criteria. Aside from that, I personally would not touch anything without remote management (IPMI, AMT, etc) anymore, it is just too convenient. Same goes for redundant power supplies, especially for VM hosts where shutdown is quite time-consuming. – user121391 Sep 16 '16 at 15:11
9

Assuming your VMs are redundant (and this is tested as working with one node off), you are probably relatively immune from an hardware related outage by virtue of having two mirrored nodes.

Without knowing more, I wouldn't recommend dropping down to a single (newer) box unless an outage of the entire node isn't a major problem at your company.

That said, it would be helpful to know some extra details about your environment... such as, how how long you've had these machines, are they in a purpose built environment (clean, dry room with a rack & AC etc). As you are probably aware, well looked after equipment lasts longer!

Generally speaking, there's nothing necessarily wrong with using less 'professional' hardware, it just doesn't come with the same guarantees or reliability as more expensive kit, and these risks need to be weighed against your budget.

Matt Renner
  • 726
  • 1
  • 5
  • 9
  • 2
    Thanks for your answer. We had the servers for 3 years now I think. They are in a rack in a dedicated room with AC. The servers are not mirrored though. Each server hosts different VMs, with each hosting one AD. – Martin Walter Sep 15 '16 at 09:27
  • In that case (it sounds like you're doing a good job of looking after them) then I think you're pretty safe to continue as you are now (assuming AD is the only critical service you're hosting on them). I'd recommend implementing some sort of monitoring to keep an eye out for hardware errors (I'm not sure what's most appropriate for Windows these days) and perhaps plan on replacing them or coming up with a different solution in the next 2-3 years. – Matt Renner Sep 15 '16 at 09:41
  • 4
    @MartinWalter Maybe you should be worried. One system dies, it's services are gone. How will that effect business? How fast can you recover? – Ryan Babchishin Sep 15 '16 at 10:13
  • 1
    @RyanBabchishin That question is always an important one for in-house IT. However, realistically there's always trade offs, and to the business, going down for an hour or two might be acceptable on that rare occasion. Also, if OP has his VM's snapshotted/exported to an external device, recovery is as simple as mounting that external device on the other running box, and booting the vm's off it temporarily. For AD, this likely won't cause any issues (if all it's handling is basic AD stuff, nothing advanced) other than reduced performance until recovery of the primary machine. – SnakeDoc Sep 15 '16 at 14:56
  • 1
    @SnakeDoc That's why it's a question... to him. About that. ? – Ryan Babchishin Sep 15 '16 at 15:44
5

Since your storage backend is all-flash, your hardware is totally OK for the mentioned workload. The only concern I have regarding your configuration that your VMs are split and running on a single server instead of being mirrored/synchronized between servers especially if they are identical. Thus I would strongly recommend you to use some software-defined storage (virtual SAN) that will let you join both servers into a single cluster and making your virtual machines immune to possible hardware failures.

Possible options are HP VSA http://www8.hp.com/us/en/products/storage-software/product-detail.html?oid=5306917 or EMC Unity VSA https://store.emc.com/us/Product-Family/EMC-Unity-Products/EMC-Unity-VSA/p/EMC-Unity-Virtual-Storage-Appliance which is free but as far as I know not allowed for production. Since you are using Hyper-V a perfect option for you would be to use StarWind Virtual SAN https://www.starwindsoftware.com/starwind-virtual-san that runs natively on top of windows and allows you to seamlessly create a fully-functional Microsoft Failover Hyper-V cluster using only directly attached storage.

I would also recommend using VEEAM B&R https://www.veeam.com/vm-backup-recovery-replication-software.html that has a free version or Bacula http://blog.bacula.org/ to backup your VMs instead of using native Windows 2012 Server Backup since it is known for causing issues when trying to recover your VMs.

Net Runner
  • 5,626
  • 11
  • 29
  • 1
    Thanks for the advice. I know backup is something that I have to address. Failover is not possible right now, because the machines are running different VMs and don't have that much RAM, so I can't put all VMs on a single machine. – Martin Walter Sep 15 '16 at 12:20
  • 1
    Would love to hear more about the issues you've mentioned about restoring VMs with Windows 2012 Server Backup – wandersick Sep 21 '16 at 01:30
  • 1
    @wandersick A common issue actually. Windows Backup is known for spawning strange errors like 0x8XXXXXXX during restore making the restoration impossible. Strange thing is that sometimes there error codes are not even googlable :-( – Net Runner Sep 21 '16 at 13:39
0

In small deployments, it is generally better to have multiple (at least 2) cheaper machines than one expensive one. Or in other words, in small deployments it is better to be wide than tall. The reason is that this way you have some redundancy for a limited increase in cost. Two 3000,- servers might be able to do the same thing as one 5000,- server, but if the expensive server fails you are boned. If one of the cheaper ones fail you at least have half of your VMs still running, and it can probably run the other ones as well, it's just going to be slow.

Something you should look into is to not manage these servers individually, but somehow cluster them together. Your virtualization solution should be able to create a failover cluster so that it doesn't matter on which host a VM lives, if the host dies, the VM is automatically migrated. This also reduces micro management and means that in the future, you can just add a new server while keeping the old ones running, until it is no longer economical to do so. This decision usually comes down to power consumption or space limitations.

If you want to grow more, you probably want to migrate from on-server-storage to a SAN. This way, your servers become pure computing nodes, and their health doesn't really matter to the VMs.

mzhaase
  • 3,778
  • 2
  • 19
  • 32
  • Be advised that with this solution/general path, there is a certain level you have to achieve for it to be correct. You can get redundancy inside the expensive machine (PSUs, disks, controllers, GPUs, CPUs, network cards) as well, while reducing complexity in management (software updates for second host, automatic VM failover orchestration, network traffic considerations, permissions, monitoring) a bit. It is difficult to say what's better without hard numbers, though. – user121391 Sep 16 '16 at 15:05
-8

as long as you don't have any problems on your network and on your back up system you can continue with this setup but for future it is better to have Professional server to be 100% HEALTY.

Shan't Baghdikian
  • 1
  • 2