16

We're speccing up some 10GbE switches for integrating a few older servers into our Equallogic SAN, and we're noticing quite a price gap between SFP+ and Copper (Cat 6A) equipment (Dell 8024F vs 8024).

I'm not really sure what the real-world difference is between the form factors. The Dell guys tell me that SFP+ has a lower latency, but couldn't tell me much more than that besides that our M1000e and PS6010XV chassis's only comes with SFP+ uplinks (and SFP+ is substantially cheaper).

Mark Henderson
  • 68,316
  • 31
  • 175
  • 255
  • 10 years later I can say, unequivocally, the most important difference between these two standards is power consumption and thus heat generation. – Mark Henderson Jun 15 '20 at 06:54

5 Answers5

12

The latency is basically negligible. The 10GBASE-T has a latency of <1microsecond. SFP+ has less latency in itself, but SPF doesn't include some of the physical transceiver (which may or may not add latency); hence the need for a physical module (or direct copper cables).

The biggest differences are price, as you've noted, and distance. SFP+ Direct Copper the cables have to be <15m (10m for certain cables). 10GBASE-T goes the standard 100m. Cat6 cabling is quite cheap (compared to other 10G cabling), and I would suspect that the equipment manufacturers "make up for that" in the price in addition to 10GBASE-T not being as popular yet.

The 10GBASE-T standard also uses more electricity (which causes both the increased distance and latency). The extra amount used isn't normally a factor.

Skyhawk
  • 14,149
  • 3
  • 52
  • 95
Chris S
  • 77,337
  • 11
  • 120
  • 212
  • Thanks. We'll be going distances of < 3m so length definately isn't an issue. – Mark Henderson Oct 27 '10 at 02:23
  • If distance happened to be a factor, SPF+ can also use Fiber to get much farther than 10GBASE-T anyway. I suspect that as 10G slows works into "consumer" hardware it will become substantially cheaper than SPF+; as 1000BASE-T is now compared to SPF. – Chris S Oct 27 '10 at 02:27
  • Agree with @Chris S, I think 10GBase-T will within 2 years be standard equipment on midrange servers (and therefore, by default, the typical interface for iSCSI SANs). – Skyhawk Oct 27 '10 at 02:38
  • 1
    Interesting. I wonder why Dell have chosen to go the SFP+ route then, as almost all their 10GbE equipment only has SFP+ interfaces... Could this be another VHS vs Betamax / HD-DVD vs Blueray? – Mark Henderson Oct 27 '10 at 03:17
  • Sort of? I don't think SFP+ is likely to disappear like Betamax. There are situations where the difference in latency actually does matter. But, 10GBase-T switches and NICs will keep getting cheaper, and the cables will stay cheap. Most importantly, 10GBase-T NICs and switches, and CAT6 cables, are backward-compatible with the 10/100/1000Base-T infrastructure that is already ubiquitous. – Skyhawk Oct 27 '10 at 14:38
  • 1
    @Miles^W @skyhawk always interesting to see how our predictions come back to haunt us. I've only seen one upgrade kit (for a QNAP) that has Base-T 10GbE. Everything else is SFP+. And certainly far from standard. – Mark Henderson Nov 12 '14 at 08:26
  • 1
    @MarkHenderson Yup. No more bold predictions from me! – Skyhawk Nov 14 '14 at 17:35
  • According to some ppl more power is an issue if you want to convert SFP+ to cat6a. You can end up with low bandwidth because SFP+ does not give enough power. – inf3rno Jun 13 '20 at 09:15
7

"SFP+ has a lower latency" due the much better noise/interference isolation, but copper is generally less fragile/more durable (if it will be in a "higher chance of being re-handled" environment).

user48838
  • 7,393
  • 2
  • 17
  • 14
  • Given the cost of this equipment, we're hoping that it will be installed and then not touched for the next 5 years or so, so hopefully durability won't be much of an issue. Of course, we all know how these things go in real life... – Mark Henderson Oct 26 '10 at 22:40
  • Yup... But the "much better noise/interference isolation" may come into play too depending on technology trends across its installed shelf life. – user48838 Oct 27 '10 at 01:57
6

The 802.3an (10GBase-T) standard calls for latency of 2.5 microseconds or better. You are dealing with storage that still has latency measured in milliseconds. The difference might matter for extremely specialized high-performance computing applications, it can't possibly have any significant impact on your SAN performance.

How does the cost difference look after you have purchased the actual cables? You may find that market prices for CAT6A patch cables are becoming almost similar to prices for CAT5e, whereas the SFP+ cabling could actually be a significant component of your project cost. (It may even cancel out the difference in switch prices.)

I would suggest that overall project cost is likely to be the most important deciding factor.

(Disclaimer: there is no 10GbE at all in my current environment.)

Skyhawk
  • 14,149
  • 3
  • 52
  • 95
  • Noise/interference isolation may in some extreme/unforeseen situations or when the specifications advance forward towards tighter tolerances. – user48838 Oct 27 '10 at 02:00
  • @user48838 I don't understand (yet). Noise/interference isolation may do what? – Skyhawk Oct 27 '10 at 02:03
  • @Miles - we've been factoring in SFP+ cables into everything as you're quite right, they're quite pricy. However once we factor in the number of switches, redundancy, NICs AND cables, SFP+ is more than $2,000 cheaper. I have no problem with SFP+, it's just not a technology I've used before, but if it's substantially cheaper and is no worse than Cat6A, then I'll go for it. – Mark Henderson Oct 27 '10 at 02:13
  • Seems like you have a winner. – Skyhawk Oct 27 '10 at 02:19
  • @Miles - would appear so, I was just concerned there might have been a big "gotcha" that I'm missing, but it doesn't seem to be. – Mark Henderson Oct 27 '10 at 02:24
  • @Miles performance growth/increase potential, if new advances come about without/minimizing the possible required technology replacement. – user48838 Oct 27 '10 at 05:51
6

Here is a couple of links talking about latency with 10GBase-t. Basically 10GBASE-T is slower than 1000Base-t or gigiabit for small packets. If you are doing something like iSCSI then it will be insignificant, but if you are doing hundreds of thousands of short key/value lookups between servers that traverse several switches it can be significant and surprising that it's slower than gigabit...

http://www.datacenterknowledge.com/archives/2012/11/27/data-center-infrastructure-benefits-of-deploying-sfp-fiber-vs-10gbase-t/

http://www.plxtech.com/files/pdf/support/10gbaset/whitepapers/10GBase-T_1000Base-T_Switches.pdf

Note: The latency is from 10GBase-T, not copper. If you do SFP with integrated twinax cable, that is copper but doesn't have the latency problems of 10GBase-T.

jlauro
  • 61
  • 1
  • 3
2

Comparing SFP+ to 10GBASE-T.

Pros of SFP+

  • Lower latency (though unless you are in HFT or similar it's likely negligable)
  • Lower power consumption
  • Cheaper NICs and switches
  • More choice of connected equipment.
  • With transcievers and fiber basically any run length can be covered.

Cons of SFP+

  • For short simplw runs needs "direct attach" cables which cost substantially more than twisted pair and are considerbally more cumbersome.
  • For longer runs or runs that need to go through patch panels needs transcivers and fiber. Fiber itself is cheap but transcievers, termination, patch panels etc for fiber an get quite spendy.

Pros of 10GBASE-T

  • Cheap twisted pair cables
  • 100m runs without messing arround with transcievers
  • Patch panels can be used without messing arround with transcievers.

Cons of 10GBASE-T

  • Higher power consumption
  • People may get tempted to use substandard cabling, just because it's twisted pair doesn't mean you can be anywhere near as sloppy as at lower speeds.
  • No good way to extend length beyond 100m (though this can be somewhat mitigated by chosing switches with mostly 10GBASe-T but also a handful of SFP+ ports)
  • limited choice of equipment.

Until/unless they can get the price and power for 10GBASE-T down it IMO has fairly limited utility.

Peter Green
  • 4,056
  • 10
  • 29