0

I've inherited two Mellanox Infiniband environments. One on a LAN, and the other in a DMZ. Both IB networks have their own SX6005 unmanaged switches and a couple small desktop computers running the subnet manager for the network. As I understand, running a managed switch would put the subnet manager on the switch, negating the need for the desktop computer subnet managers, correct?

Would it be best to purchase two switches to maintain segregation of LAN/DMZ or is the Mellanox Inter-VLAN routing "good enough"?

Also, I'm pretty new to all the Infiniband technology. How do I know which switch is compatible with the my server NICs (MCX354A-FCBT CX35A FDR)? Will any FDR switch work or does the interface type come into play as well (QSFP, QSFP+)?

Thanks

Server Fault
  • 3,454
  • 7
  • 48
  • 88

1 Answers1

0

Yes, a managed IB switch will run the subnet manager (SM), so it is not required on any system. However, opensm (the standard SM that is run) doesn't need to stay running all the time - for the most part, it's only needed to assign a LID to a node when it connects to the IB fabric. Thus, it may still be in your best interest to run opensm instead of buying a managed switch.

Since you already have the hardware, having two physically separate networks makes the most sense. You can use partitions to isolate traffic, but it can be easily misconfigured. If you're using the cards in Ethernet mode (if the cards and switch even support that), then yes, the VLAN segmentation is reliable.

Any FDR switch will work with your FDR cards. Make sure you have legit FDR cables, otherwise your cards will only link up at a lower rate, or not link up at all. The only way to verify all of this is to carefully look up all model numbers - cards, switches, cables.

Paul
  • 424
  • 1
  • 5
  • 14