0

I have the following configuration:

I------------------I                 I------------------I
I                  I(P0)---------(P0)I                  I
I  Voltaire HCA 0  I                 I  Voltaire HCA 1  I
I                  I(P1)---------(P1)I                  I
I------------------I                 I------------------I

The two HCAs are directly connected and I have managed to get opensm working to some extent. The odd thing is that I can ping to and from every interface but one. I can ping to it but not from it. Although I have significant experience in IP networks I'm somewhat at a loss when it comes to infiniband. The sole function of this is to have two NFS shares (allowing 10G bandwidth to each).

user186658
  • 1
  • 1

1 Answers1

0

If this is really your configuration (two HCAs connected to each other w/o switches), then you need two SMs running- one for P0 ports and the other for P1.

You can't run two SM instances on the same machine (well, you can, but you need to add several command-line parameters to do it), so run SM on machine with HCA0 as usual, and run another SM on the machine with HCA1 on the second port.

Don't know which SM vendor do you use and how old is your SM (Voltaire has been bought by Mellanox four years ago).

If it is OpenSM, you can run "opensm -g 0" and then select the second port.

kliteyn
  • 154
  • 2
  • Unfortunately just using that did not work. Strangely enough if I start opensm on one side, configure all four ports, and then down ib0 on each end (and up it again) all seem to work. – user186658 Apr 04 '15 at 20:32