27

This is a Canonical Question about choosing a network switch for a datacentre

When shopping for a networking switch that's going to be going into the top of a datacentre rack, what specific things should I be looking for?

i.e. What makes a $3,000 Cisco switch that requires annual maintenance a smarter buy than a $300 Netgear Smart switch with a lifetime warranty?

ewwhite
  • 194,921
  • 91
  • 434
  • 799
Mark Henderson
  • 68,316
  • 31
  • 175
  • 255

4 Answers4

30

Context is everything... There's no blanket answer.

If you're trying to ask: "what differentiates an expensive switch from a low-end switch?" or
"is there a reliability difference between a high-end switch and an inexpensive switch?"

The answers are "feature-set" and "maybe", respectively... I've used a $40,000 switch for the specific purpose of connecting two 10GbE WAN connections in a data center cabinet. I've also seen $100 unmanaged Netgear FS524 switches run the "core" of a $400 million/year company for 9 years, with no reliability issues...

"You're only using me for my 10GbE ports, routing capabilities and good looks..." - Cisco 4900M. enter image description here

If you're looking for a rule or general advice that can be applied across the board, there are more questions that deserve attention:

  • What type(s) of systems are co-located in the data center facility? - This is basic. Running a pair of web servers at a cheap colo is different than managing a full application stack or virtualization cluster in a high-end facility.
  • What is the purpose of the switch? - As above, if there are throughput, latency, buffer or other performance considerations, that's going to drive the type of equipment you use. And there are definitely switch attributes that impact the success of deployments for iSCSI, VoIP, PoE, low-latency and streaming technologies.
  • What interconnects are required? - These days, this may determine the class and tier of switch more than anything else. People want 10GbE connectivity for storage and other network workloads. Below a certain price threshold, you simply won't find that type of connectivity. Fiber? SFP+? Compatible with Copper DAC? Dedicated stacking links? HDMI?!?
  • How complex is the network? - Will these switches link back to a core? Are they the core? What's their place in the overall design? In my work environment, we use lower-end Layer-2 access switches that offload the heavy lifting to a central core switching/routing infrastructure.
  • Power - Depending on the colo/facility, power constraints, etc., redundant power supplies are a nice option. But they're not a requirement. I rarely see switch power supplies fail. But it's possible to keep a cold-spare and copies of configurations handy, too. Redundant power supplies often push devices into a much higher price bracket.
  • Cooling - Fan design, hot-pluggability and the option to control switch airflow are nice features.
  • What resiliency and redundancy options do you need? - Chassis switches, modular switches, stacked switches and standalone devices can all have different levels of resiliency. But I think their feature sets and other network design considerations tend to be more important.
  • Warranty and support - I don't buy Cisco SmartNet often enough... But the product is so ubiquitous that finding technical resources/parts/support hasn't been difficult. I think the HP ProCurve Lifetime Warranty is often overlooked. For something like Netgear, I don't know that they would provide good technical support. As stated earlier, if the cost is low enough to afford cold-spare units, you can self-support on the hardware side.
Community
  • 1
ewwhite
  • 194,921
  • 91
  • 434
  • 799
  • 2
    A photo of yet another CISCO behaving like a princess . – MadHatter Sep 09 '13 at 09:21
  • 6
    Redundant power supplies don't just protect against failure of the power supply itself. You also gain the flexibility to switch to a different power source in case your UPS has to be replaced or your electrician needs to shut down a circuit for some work. – 200_success Sep 09 '13 at 09:36
  • 1
    Epic answer, as usual – Mark Henderson Sep 09 '13 at 10:16
  • I would add monitoring capabilities: Number and type of SPAN / mirror sessions and Netflow. – Gerald Combs Sep 09 '13 at 19:27
  • Good answer. Possible additional item for resiliency/redundancy bullet point: U's of space required, number of switches required for redundancy, how they interconnect, how they are managed (e.g. VC from Juniper, etc) – dmourati Sep 10 '13 at 01:29
14

For my money, the one absolutely mandatory thing is that it be remotely manageable. Sooner or later, you'll get a duplicate IP address, or duplex mismatch, or hard-to-track-down top-talker, or some other problem that can be answered in seconds with a manageable switch. If you can't ask your switch what it thinks about packet counts, errors on ports, where the MAC addresses are, that kind of thing, and if you can't do it remotely, you'll be sitting in a data centre for hours unplugging cables one at a time, to see if the problem has gone away.

The remote manageability should be via CLI, not just a stupid web browser, because the networking gods will cause your switch to fail when you're in the middle of nowhere, and only able to connect to the DC over a slow EDGE connection, via a stupid web proxy that insists all graphics are bad.

After that, it depends on whether the switch needs to be configured or not. If it doesn't need significant configuration (no VLANs!) then anything that's half-decent will do, because when it fails, you'll replace it with whatever seems nearest to half-decent at the time.

If configuration is required, there's definite value to buying a long-living, UI-stable brand like CISCO, because you're most likely to be able to take config for the old switch out of your config repository and blow it onto the new switch with minimal problems (another reason why a CLI is good; web configs can't be trivially saved in, or restored from, a repository).

The final thing to consider is modularity. I've just decommissioned a data centre deployment that I built and installed over ten years ago. We went for very expensive, large (4U) modular HP switches, so that if one failed, we could replace the faulty blade without downtime. During those ten years, no blade failed; the modular switches were a waste of time and rack space. If I had it to do over again, I'd have used high-port-density switches instead; still high-quality, to minimise the chance of failure, but now you can get 48 ports in 1U, it's an odd rack that needs more than 96 ports, even with dual-connect for everything.

MadHatter
  • 78,442
  • 20
  • 178
  • 229
  • 14
    And if you do have to use a Web-based management interface, under no circumstances should it require Java. I'm looking at you, Extreme. – Tom O'Connor Sep 09 '13 at 10:04
  • 1
    We have Dell PowerConnect 2824 switches, and it pains me that its CLI is crippled so. Its web interface is near incomprehensible at first glance and its attempts at reducing complexity actually make the situation even more complicated. – Aaron Mason Sep 10 '13 at 01:22
4

Depending on the application, power consumption may also matter. Power in a colo space can get expensive fast, and you don't want to use a 250W switch (e.g. Procurve 6600-48G) where a 50W switch will do (e.g. Procurve 2920-48G).

gwf
  • 361
  • 1
  • 3
  • 8
3

There are a lot of difference between a $3000 and a $300 switch (ex.):

  • redundant powersupply
  • managing acces
  • vlan support
  • support from manufacturer/distributor
  • stackable (for example with FC)

Just my 2cents.

deagh
  • 1,969
  • 5
  • 18
  • 16