Direct connect 10GBE with fibre

Question

I have two servers that I wish to direct connect with 10GBE.

One server has an Oracle Sun 10GBE (Intel 82599) with dual SFP+ pluggable transceivers installed. The other has an HP 10GB MELLANOX CONNECTX-2 with no transceiver installed.

I'm planning to move one transceiver from the Sun to the Mellanox.

Can I direct connect the two interfaces with fibre, and if so what kind of fibre cable? If not, what kind of cable do I need?

score 6 · Accepted Answer · answered Feb 25 '16 at 12:07

6

You may be better off just using a copper DAC cable. But otherwise, you probably just need a multimode fiber LC-LC cable of the appropriate length.

answered Feb 25 '16 at 12:07

ewwhite

194,921
91
434
799

I'll just add to this that you'll need to "cross over" the fiber connections on one end, just as you used to have to do with 10/100 ethernet. – Mark Henderson Feb 25 '16 at 14:43
Could you please elaborate on why copper DAC might be better? Do you more likely to just work? – retrodev Feb 25 '16 at 15:30
LC-LC cables seem to listed as simplex, duplex, single mode and multi mode. Any suggestions there? – retrodev Feb 25 '16 at 15:32
DAC, because you don't know if your Sun transceiver will work on the HP card. – ewwhite Feb 25 '16 at 15:34

score 2 · Answer 2 · answered Feb 25 '16 at 12:08

2

I'd recommend using Direct-Attach Cables: this is sort of semi-active equipment that consists of a cable and two strictly attached SFP+ transievers at it's ends. I doubt plugging Sun transiever into Mellanox adapter will do any good (but you can try, and I may be wrong), and DACs are cheap. The only thing to consider is the distance between these servers - unless they are around several meters one from another getting a DAC of this length may become impossible. Another thing to consider - flashing the transievers on the ends with a proper [vendor] firmware, but this is usually taken care of by the DAC vendors.

answered Feb 25 '16 at 12:08

drookie

8,051
1
17
27

So are transceivers normally vendor specific then? – retrodev Feb 25 '16 at 15:30
Only by their firmware: there are only few companies if the world that manufacture SFP/SFP+ transievers, but every vendor tampers them with terir custom firmware, so people have to buy only so-called *"original"* transievers. Otherwise they would be detected as some "general" ones or even, in worst-case scenario, won't be supported at all. – drookie Feb 25 '16 at 15:50

score 0 · Answer 3 · answered Jun 07 '16 at 09:05

DAC cables is perfect for your current situation. SFP+ DAC cables, as the name implies, use SFP+ MSA and the inexpensive copper twinax cable with SFP+ connectors on both sides, provides 10G Ethernet connectivity between devices with SFP+ interfaces, which make itself a cost-effective solution for 10G network application.

score 0 · Answer 4 · answered Jan 28 '18 at 00:49

DAC SFP+ over copper works well. Have configured A B C Windows 10 -cable- Windows 2012 -cable- Windows 2012 Middle machine has 2 10G NICS, bridged. Others have 1 each. (All purchased on eBay for around $20 each).

All machines also have 1G NICs to access internet and serve HTTP. 10G NICs form a privatee network with IPs: (A) 10.1.1.1, (B bridge) 10.1.1.2, (C) 10.1.1.3. Achived 500-700 MB/s between two machines directly connected, and 250MB/s between A and C (i.e., traffic had to route through software bridge in B).

Direct connect 10GBE with fibre

4 Answers4