1

Good day,

and my disclaimer that I am no IB expert and have not worked on IB directly yet. My question seems a trivial one, yet I could not find anything in google that answers it.

Question: In Infiniband can I use the same port for IPoIB traffic as well as RDMA traffic? Or can the port (interface) only be used for one communication method at one time? And if yes how does the IPoIB and RDMA prioritize to each other?

Background story: I have one server with one dual port FDR IB card connected to two Mellanox switches accessing the storage system in RDMA (performance) and at the same time providing SMB windows shares over IPoIB over the Mellanox switches which have an Ethernet gateway and Ethernet breakout cables to connect to Windows clients.

Server: connects to storage in RDMA and Windows clients in IPoIB providing SMB shares using same Ports via Mellanox switches for both (RDMA and IPoIB)

Storage: provides multipath storage LUNs to servers in IB RDMA mode via Mellanox switches to server

Mellanox switches: export Ethernet connectivity vie ethernet gateway and ethernet breakout cables for IPoIB to Windows clients

Windows Clients: using 10Gbit Ethernet to access windows share on server

Many thanks in advance for your kind help.

BeGa
  • 21
  • 6
  • If you also use other IP protocols that don't have RDMA support some Mellanox cards can be switched to pure ethernet mode (with RDMA also supported). This also requires switch support. The advantage is you avoid the IPoIB CPU overhead. RDMA negates that though. – hookenz May 18 '17 at 04:49

2 Answers2

2

Absolutely, you can have IPoIB and RDMA on the same InfiniBand port.

Regarding prioritization, InfiniBand can support multiple traffic classes. You can find information in this document: http://www.mellanox.com/pdf/whitepapers/deploying_qos_wp_10_19_2005.pdf

haggai_e
  • 272
  • 1
  • 7
  • the document looks interesting will have a a read later. Is it still valid though from 2005 (which is ages ago in IB world)? – BeGa May 21 '17 at 09:26
0

The architecture for QoS did not change. Note that you will need SubnetManager to configure the different QoS levels. See OFED OpensSM doc http://downloads.openfabrics.org/OFED/archive/ofed-1.4/OFED-1.4-docs/QoS_management_in_OpenSM.txt

user416443
  • 1