When the lights go out, do your servers too?
I live in an old house with old wires in an old town with a dated infrastructure. I've never really considered the quality of the electricity being pumped into my house, but since we moved to this charming house loaded with character (that's the nice way of saying “old”), it's something I've had to deal with often. I made a lot of assumptions and even made some expensive purchases that in hindsight were silly. So for this article, I figured I should talk about power, because if you're reading Linux Journal, you're familiar with things that plug in to the wall.
The biggest red flag for me in the new house was my Internet connection behaving poorly. I've literally had the technicians out here more than two-dozen times trying to track down why I have massive packet loss. At first it was unusable (50% packet loss is something TCP/IP can't even error-correct), but after months of struggling, I have only minimal loss. Unfortunately, my day job requires me to have a solid Internet connection, so I've been forced to install two separate connections and provide failover support if (when!) one of them goes down. Although the Internet connections may be the most sensitive to electrical issues, I know that unstable voltage and noise can be bad for all my servers as well.
Quite honestly, even if you're not having Internet issues, there are some things to look for that might be symptoms of power problems. Now that I'm paying closer attention, I notice that a few times a day, our lights flicker or quickly dim. I assumed at first it was when a large motor kicked on (causing a sag) or something like that, but now I don't think it is. It appears to be unrelated to anything in the house, and it happens often enough that it's not just a fluke. If you notice tiny glitches in the Matrix, it might be a sign that your power fluctuates.
Even if your house's electrical service is rock-solid, it's very important to have your electronic equipment on a surge protector. Although they used to be expensive, you can get a really nice one for less than $20 at most stores. A surge protector's job is to protect your equipment from current fluctuations, but only extreme fluctuations or surges (thus the name). A surge protector won't generally help with minor issues like those in my house, but during a lightning storm, you'll be glad you spent the money.
If you live in a place that rarely gets lightning storms, or if you just don't think a lightning strike is likely in your area, I still highly recommend a surge protector, or surge suppressor as they're often called. When I worked at a local school district, there was a failure on one leg of the three-phase power feed that caused a huge surge to enter the building. It was a sunny day without lightning, and still the surge destroyed thousands of dollars worth of non-protected equipment.
The local cable technician recommended I purchase a power conditioner in order to give my equipment clean power. This actually started my research on power, and before I did much studying, I ordered a rack-mounted power conditioner for all my servers. I wish I had done more research before ordering it, but it does serve a purpose, and what it does, it does very well.
A power conditioner does a couple things. It protects your equipment from surges. In fact, a surge protector is a power conditioner of a sort; it's just the most basic kind. The other main purpose of a power conditioner is that it “cleans” the power signal by filtering out electromagnetic interference (EMI) and radio frequency interference (RFI) that can really mess with sensitive equipment. I've actually never had problems with EMI/RFI on server equipment, but if you have any audio equipment and you hear a buzz, or odd noises in your recordings, a line conditioner might really help.
The model power line conditioner I bought has a really cool display on the front that shows the line voltage. In fact, although it doesn't do anything to help with voltage fluctuations, it is nice to have the readout showing me as voltage comes and goes. Figure 1 shows my power conditioner. It's in my office now, but I can see the voltage vary anywhere from 103V to 124V throughout the day. That's a huge variance, and it's very likely what has been causing part of my Internet problem.
Figure 1. I love the readout on this line conditioner. I just wish it were a voltage regulator!
Although the power line conditioner is great at identifying voltage issues, it doesn't actually do anything to fix them. Unfortunately, the tool to normalize voltage is quite expensive. I paid around $400 for a rack-mounted voltage regulator. The good news is that most voltage regulators not only regulate the voltage, but they also condition the line and protect from surges. (But you really should read the fine print to make sure.)
The voltage regulator on my server rack functions just like a power strip. It plugs in to the wall, and then I plug my devices in to the back. All the devices see a perfectly steady 120V power source, however, even if my actual house voltage gets wacky. It's not a magic device, and it won't work in the case of a brown out, but it can handle voltage sags down to 95V, which is lower than I've ever seen it drop in my house.
The specific voltage regulator I purchased is the Furman M-8X AR (Figure 2). It supports up to 15 amps of service, which is all my server rack power line is rated for anyway. I do wish it had a clearer readout for incoming voltage like my line conditioner does (the line conditioner is the Furman M-8Dx), but that doesn't affect its performance, just my desire for data!
Figure 2. I don't regularly look at the front panel, so really this readout is sufficient. I just prefer the large LCD numbers showing incoming voltage.
That last thing I want to touch on here is a battery backup. There's a lot to be said regarding the type of battery backup you use, but I just cover them briefly here.
Standby UPS:
This is the type of UPS most folks use, even in server rooms. There is a battery, which is kept charged by the main power line, and if the power goes out, it quickly powers an inverter to supply 120V current. This is very efficient, because the inverter doesn't constantly convert the battery power into AC, it only “does work” when the power goes out.
The downside of all this efficiency is that there's a small cutover time when the line power is disrupted before the inverter takes over. It is usually measured in milliseconds, and the more expensive the standby UPS, the quicker the cutover time. This is typically not a problem for most computer hardware, but particularly sensitive computers may reboot during switchover.
Standby-Ferro UPS:
This type of UPS functions in the same way as a standby UPS, except there is a built-in transformer that regulates the voltage and provides a bit of a “buffer” during the cutover to inverter power. Since transformers use magnetism while converting voltage, there is a residual magnetism that usually smooths over the transition so that even sensitive equipment doesn't suffer.
Since they have built-in voltage regulation, they help with fluctuating voltage even when there's not a power outage. Unfortunately, they are known to overheat with some generators, especially those not generating pure sine waves. Still, with the buffering effect of the voltage regulation circuitry, Standby-Ferro units are usually preferred and usually cost significantly more than standard Standby UPS units.
Line Interactive UPS:
These types of backups provide the smoothest transition during an outage, because they constantly use their inverter to produce electricity. The only switchover is whether the inverter is taking power from the batteries or the wall current, and since both are connected, there's not even a millisecond of cutover time to worry about.
These are the most expensive types of UPS devices, and although they're very nice for sensitive equipment, they're often overkill for protection from the occasional outage. Still, if you're looking for the best of the best and price is no object, a Line Interactive UPS is the Rolls Royce of the UPS world.
I have terrible luck with UPS devices. It seems like the batteries are always dead when our power goes out, and I have no idea until it's too late. And then, even the fairly expensive rackmount units are often disposable without the ability to change their batteries. Even if the batteries are replaceable, and you actually know they've gone bad, UPS devices don't last very long. Usually the best you can hope for is to keep your servers up long enough that they can shut down properly instead of just losing power. That's a great benefit, but it's frustrating if you want to function during an outage.
For my home, I decided to separate the inverter from the battery. I bought an AIMS inverter/charger unit (Figure 3) that is hard-wired inline with my house wiring. The specific unit I bought is the 3000-watt pure sine inverter. It gives clean voltage and uses external batteries. In my case, it uses a huge 12v marine battery. This will keep my little server rack running for quite some time before the battery is drained, and since it's a robust lead acid battery, it seems to last better than the tiny rackmount UPS batteries. Time will tell.
Figure 3. I really like having a separate inverter/charger and battery. If the battery dies, I can replace it without replacing the inverter hardware itself. Plus, solar power will be fun to play with!
I decided to go with 3000 watts because I wanted to make sure the full 15 amps of service would be supported, even though I don't use nearly that much on my rack. I also like the AIMS power inverter because it comes with connections that support charging with a solar array along with grid power. I have no intention of going “off grid” with my unit, but I love the idea of offsetting a little of my electrical bill with solar panels. (That will likely be a future article!)
Quite honestly, my power problems are likely more pronounced than most folks. In our last house, I never had any problem with our line power at all, and the idea of line conditioners and voltage regulators seemed absurd. With our new house, however, if I want to be able to work from home and use computers all day, I have to make sure my power is rock-solid, and my two Internet connections are reliable. But who knows, maybe I actually will decide to go off the grid someday. If that ever happens, I'll be sure to write about solar panels and battery banks. Until that time comes, enjoy your conditioned electricity!