52

According to this comment by Tom O'Connor (slightly edited below):

You can seriously cheese off a datacentre by putting an UPS inside your own rack.

What are the risks to the data center should a customer (somehow) choose to do this?

JonathanDavidArndt
  • 1,414
  • 3
  • 20
  • 29
  • 9
    Things don't happen in a vacuum. If your datacenter prohibits the use of a rack UPS then they'll either prevent you from installing one in the first place or they'll require you to remove it. – joeqwerty Aug 27 '18 at 14:09
  • 3
    Yes, I certainly have no intention of doing this myself. Not knowing many of the edge details regarding server setups, I was simply curious as to why someone would think this was a good idea, and what risk this could pose to a data center. – JonathanDavidArndt Aug 27 '18 at 14:43
  • I don't have the maths for this, but there is a thing called Power Factor, where the characteristics of the load affect the supply. A UPS has a different power factor than the load. https://en.wikipedia.org/wiki/Power_factor Someone with better skills can do a proper answer. – Criggie Aug 28 '18 at 02:14
  • 2
    Not all datacenters are created equal. Not all datacenters will have the same rules regarding the use of a UPS inside your cage. I've worked in datacenters where all the points given by others in this thread are valid and a UPS should never be used. I've also worked in lower-budget datacenters with less reliable power than rural Afghanistan and the UPS was welcome and useful. – Ryan Ries Aug 28 '18 at 13:53
  • Sometimes you can request unconditioned generator power. I used to have some equipment in a coloc with a kinetic backup power system (i.e. a flywheel) that would power the datacenter just long enough for the generator to start (20 seconds? "If the generator doesn't start by then it's not going to start at all"). One large customer didn't trust that system, so they had their own large UPS setup in their cage (which was connected to the facility EPO button). I was in that coloc for 2 years, and it survived several power failures without issue so I don't know if the concerns were warranted. – Johnny Aug 29 '18 at 20:17
  • Is this a datacenter or merely a colocation point? The former has generators, the latter may not. – Criggie Jul 15 '21 at 19:28

6 Answers6

81

Pros:

  1. None

Cons:

  1. It interrupts the flow of the Emergency Power Off (EPO) in the datacenter. If there is a life or death emergency in a datacenter, that EPO might be triggered to save someone's life. If your rack has its own UPS, it will violate that EPO order.

  2. You will not get extended runtime. Chances are as soon as your upstream UPS switches into battery mode, your UPS will detect the change in sine wave and drop into backup mode as well.

  3. You'll violate your warranty and potentially your datacenter's UPS warranty. UPSs are warrantied to be installed in very specific scenarios and power sources. Your UPS-on-UPS is not going to be a supported configuration.

  4. (from rexkogitans) You have to care about your UPS while there is a staff caring for a UPS that may be provided to you nonetheless. So it adds administrating a UPS to your work really unnecessarily.

Mark Henderson
  • 68,316
  • 31
  • 175
  • 255
  • 6
    Can you elaborate on how this could create or contribute to a life-threatening situation? – R.. GitHub STOP HELPING ICE Aug 27 '18 at 16:09
  • 28
    @R.. The most obvious to me is if a faulty power supply cause an employee to get electrical shock. If a colleague notice this soon enough and cut power to the rack it may prevent this being fatal. But if there is an unauthorized UPS that could mean the electrical shock is not mitigated by powering off the rack. The other possibility is in the event of a cooling failure it may be necessary to cut power to prevent fire and/or damage to equipment. If there were several unauthorized UPS that wouldn't be an effective mitigation. – kasperd Aug 27 '18 at 16:21
  • 1
    I know about an accident a while back. Some guy mistakenly opened a valve when checking water pipes and topped few cubic meters of water into the datacenter which was in the basement of the same building. So... water pool + electricity. – Fiisch Aug 27 '18 at 16:24
  • 8
    @R.. yeah pretty much was kasperd said. If the datacenter has a serious enough issue that they need to hit the EPO button, then they need everything _off_. If your rack is still there consuming 30A of 240v power after they think everything is off then you can contribute to injury. – Mark Henderson Aug 27 '18 at 16:47
  • 2
    @kasperd: Ah. I always use low-voltage DC-DC UPS's, so I wasn't thinking about the possibility of having mains-level output from a UPS. – R.. GitHub STOP HELPING ICE Aug 27 '18 at 17:18
  • 2
    @R.. A low voltage does address some of the concerns. Other concerns can be addressed by using a battery which only the capacity you need for a clean shutdown and not a big battery that can keep your server running for many minutes and maybe even an hour. And more concerns can be addressed by discussing your plans with the datacenter before you implement them. – kasperd Aug 27 '18 at 17:41
  • @kasperd: Yes, of course there's also flammability/chemical energy capacity which are hazards and your advice is all good for someone wanting to use the kind of setup I do in a data center. I think the interference with EPO to save someone's life is covered by not using high voltage, though; that was the only aspect I questioned. – R.. GitHub STOP HELPING ICE Aug 27 '18 at 21:32
  • 2
    @R.. Besides electrocution, an electrical fire is another scenario where EPO would be useful. If the electrical fire is supported by a UPS, then using EPO to help contain it won't work as well. UPSes themselves contain components that could cause an electrical fire if they fail in the worst possible way, or they can emit smoke on failure which could unnecessarily trigger fire warning and suppression systems, plus they generate a non-trivial amount of heat which adds to the thermal load on the overall datacenter. – Todd Wilcox Aug 28 '18 at 13:56
  • 1
    Having had a regular power strip (no battery involved) light on fire when we had some power problems in our *house*, I would not want one in a datacenter next to my rack that someone else was responsible for. I don't know how well they maintain their gear. – Wayne Werner Aug 28 '18 at 23:47
  • 1
    From personal experience (not datacenter), an own UPS allows to connect it to the server with a data cable and allow the server to shut down gracefully when the battery runs low. One would think there must be solutions to pass a shutdown signal to all connected equipment, but I've never, ever seen anything like this. – ivan_pozdeev Aug 29 '18 at 04:12
  • Another Con: You have to care about your UPS while there is a staff caring for a UPS that may be provided to you nonetheless. So it adds administrating a UPS to your work really unnecessarily. – rexkogitans Aug 30 '18 at 12:47
39

Apart from low-probability scenarios, the single most important reason an unnecessary UPS really cheeses people off is that a year or two from now, it will start beeping.

An UPS needs regular battery replacements, and it usually tells about it by making a really annoying beeping noise that goes on forever. In a locked-up rack. Right next to the other rack you are trying to get some work done on. And it will keep making that noise for days (or months), because whoever installed it and didn't maintain it, also couldn't be bothered to configure any remote alerts on it.

And adding insult to injury, all this happened, because someone wouldn't trust the much better UPS system already in place. Yes, use an UPS when you have to place your rack in a utility closet of an 18th century farm house. But don't do it, when you have way more reliable power readily available on the outer side of the UPS.

Bass
  • 601
  • 4
  • 8
17

You may need to run your own UPS when your datacenter doesn't quite deserve the name and it does not provide centralised UPS for you and other customers.
On the other hand, if that is the case there will probably not be a rule preventing you from installing your own though.

If you only get a couple of units in the top of a cabinet, good luck hoisting your UPS that high and not tipping over the cabinet, those things can be heavy!

And maybe stating the obvious: an implied consequence when you need to run your own UPS, is that everything that is not connected to it will not be protected. That may include your connectivity and other services the datacenter provides.

HBruijn
  • 72,524
  • 21
  • 127
  • 192
10

Don't forget the Electrical Code. Most have something like NEC 110.3b, "equipment must be used according to its labeling and instructions".*

That gives the instructions the same force of law as the Code itself.

If your UPS says "do not feed from a UPS" or if the datacenter's UPS says "do not supply UPS's", then you place the datacenter in dutch with the electrical codes. You don't get written up - they do.

* Because that is the scope of testing that was done when it was UL listed. This section is partner to 110.2, which requires only approved equipment be used. All this is on the first page of NEC. You can't miss it.

Harper - Reinstate Monica
  • 219
  • 1
  • 4
  • Um, where in the US legal code does it say that I have to follow the label on an electrical device? – zymhan Aug 28 '18 at 18:24
  • 6
    @zymhan in the electrical codes, which are cast at the state or community level. In the US, nearly every single jurisdiction *incorporates NEC by reference*. Technically NEC is a "model law", authored by a nonprofit, and offered to any government which wants to make it their actual law, with or without local customization. Most jurisdictions say "we adopt NEC 2014 as law, with the following changes: a b c". Congress ratifies the bill, sends it to the governor, he signs it, and NEC 2014 (as modded) is the law of the land. – Harper - Reinstate Monica Aug 28 '18 at 18:33
  • @zymhan - It likely doesn't. People have sued (and won) with much less than the explicit letter of the law. Precedent matters almost as much as the law itself. – Dan Esparza Aug 29 '18 at 11:05
  • If you construct a building, sure you could be criminally liable for violating the code. But no random person is going to be held criminally liable for connecting a UPS in a datacenter. And being sued has nothing to do with criminal law. But this is a computer forum, so that's neither here nor there. – zymhan Aug 29 '18 at 21:25
2

Most rack-style UPS units have, according to their label specifications, significantly higher input wattages than the equipment they are used to feed. No matter what alternate calculations or measurements or precautions YOU took, a third party planner or inspector HAS to assume that the power feed to such a unit needs to be dimensioned for what is on the label of that unit.

Also, while you might say "worst it can do, should it unexpectedly draw all of its rated power, is trip an overcurrent device, which is rated for the ampacity of all wiring behind it" - in a multi-tenant situation, this happening can affect other customer equipment. And every unscheduled reboot or disturbance of customer equipment, even if it is still within SLA tolerances, drastically worsens that customer's satisfaction with the provider.

TL;DR for next two paragraphs: Corrosive liquids, flammable gasses, electrocution hazards.

Also, all common battery systems are a chemical hazard - outgassing (lead acid batteries can generate flammable gas if a charger fails in the wrong way), leakage and subsequent corrosion (in the worst case of someone else's equipment. Lead-Acid batteries contain sulfuric acid, which is the very definition of a corrosive chemical!), fire (especially with new-school lithium systems that seem to be entering the market - also, a fire safety system designed to deal with an electrical or battery gas fire might not be designed to deal with a metal fire!)... if you bring in equipment of a brand/model/design and/or in a condition that differs from what the personnel in that datacenter knows well in practice, it will mean an unknown risk and extra training requirements.

Also, most UPS units can easily form an "IT" type mains on their own by accident - which presents an electrical hazard. This has nothing to do with IT, I am referring to "isole terre", meaning a floating-ground AC mains supply that could cause severe injury without any RCD type device interfering!

Personnel that is well experienced and trained in both types of hazards with their own choice of equipment could have a hard time dealing with your equipment if there is any incendent - not knowing eg how to shut it down NOW and RELIABLY if there is an electrical accident downstream of the unit, or how to quickly and safely remove a battery that failed catastrophically.

rackandboneman
  • 2,487
  • 10
  • 8
1

I can think of two things that this might help with, and that is providing you with the ability to network control your own power to nodes for the purpose of on-demand full node shutdown/poweron and power-based fencing for clustering.

Don't do it, though. If your colo does not offer these features and you need them, go elsewhere with your business.

Spooler
  • 7,016
  • 16
  • 29
  • 7
    Please note that you don't need a fully fledged UPS for remote power control, a switched power distribution unit (PDU) is sufficient (and a lot cheaper per outlet as well) – HBruijn Aug 27 '18 at 15:19
  • If it's an actual server hardware, won't it have power control via the BMC already? – user1686 Aug 27 '18 at 19:04
  • If that is configured, yes. That's a better way of doing things in most cases. In some colos, customers don't have access to the BMC of the machine they're running on. There are a few reasons why you might not have IPMI, but it's generally preferred. – Spooler Aug 27 '18 at 20:32