Questions tagged [infiniband]

InfiniBand is a switched fabric computer network communications link used in high-performance computing and enterprise data centers. This tag should be used for questions about IB related hardware and software.

InfiniBand(IB) is a switched fabric computer network communications link used in high-performance computing and enterprise data centers.

Its features include high throughput, low latency, quality of service and failover, and it is designed to be scalable. The InfiniBand architecture specification defines a connection between processor nodes and high performance I/O nodes such as storage devices. InfiniBand host bus adapters and network switches are manufactured by Mellanox and Intel.

In addition to hardware manufactured by Mellanox and Intel there are a variety of software stacks available for the configuration and management of IB fabrics. Most major Linux distributions have some built-in support for Infiniband technologies, and the Open Fabrics Alliance(OFA) maintains the Open Fabrics Enterprise Distribution(OFED) which provides many software and firmware packages for the management, maintenance and setup of IB networks. In addition to the 'official' OFED maintained by the OFA, Mellanox and Intel both produce software stacks for their products that typically include their own customized versions of OFED.

108 questions
1
vote
1 answer

How do I get IP packats forwarded/routed to/from my Infiniband network

I have two networks. One is a standard Ethernet network running IP. The second network is Infiniband which in addition to some custom protocols can talks IPoIB (IP over Infiniband). The router that sits on this network can ping both the Infiniband…
hookenz
  • 14,132
  • 22
  • 86
  • 142
1
vote
2 answers

Alternatives to bridging infiniband ipoib within xen domains?

I've installed xen-hypervisor-4.1 package onto an Ubuntu 12.04.3 server. It has an infiniband adaptor installed and I run IPoIB with it. What I'm wanting to do is provide the ipoib to one or more guests. However using linux bridge is not…
hookenz
  • 14,132
  • 22
  • 86
  • 142
1
vote
3 answers

are DDR infiniband and 10Gbase-CX4 cables the same?

They looks the same. It's SFF-8470 Is it ok to buy a "10Gbase-CX4" cable for an infiniband network ?
user1219721
  • 467
  • 1
  • 6
  • 15
1
vote
2 answers

How to simulate high video payload to benchmark an SSD?

I'm trying to evaluate some SSDs for a video on-demand use-case. We've done some benchmarking tests on them, but we'd like to get an idea of the number of video streams that they can support with a more realistic load test than typical benchmark…
1
vote
1 answer

NFS performance trouble: with infiniband gets stacked with many users?

I have a RAID system on A server and similar configuration on B server-both with Scientific Linux 5.5. They are connected with infiniband and RAIDS from A are mounted with NFS to ServerB. Now when the users are accessing A-Raids via B-Server, the…
Arman
  • 555
  • 2
  • 8
  • 18
1
vote
1 answer

Infiniband QDR to DDR

I have a bunch of computer with onboard Mellanox ConnectX-2 QDR Infiniband 40Gbps Controllers. They have QSFP connectors. I have a switch with 24 4x CX4 DDR connectors. If I buy QSFP to CX4 cables, will the QDR controllers on the computers be able…
1
vote
1 answer

HPC master node no infiniband network influence on compute nodes - Slurm management

I'm writing because I'm facing an issue that I cannot solve trying to configure a cluster with a master node ( or Frontend node ) as a Virtual machine managing nodes with infiniband network. I use slurm on this nodes, the frontend node is the slurm…
SimoneM
  • 11
  • 1
1
vote
1 answer

Infiniband OpenSM N-to-N port routing configuration

I have 10 servers with two CPUs each and one Mellanox 100G Infiniband NIC per CPU. Each NIC is connected to a single Mellanox 36 port 100G IB switch. My RDMA application runs as one process per NUMA node and binds to the local NIC to avoid cross CPU…
Hugo Maxwell
  • 121
  • 4
1
vote
1 answer

Unable to get Mellanox ConnectX-3 to work with transceiver/cable

I'm attempting to upgrade my local network setup to 40G but I seem to have hit a snag. I have Mellanox ConnectX-3 VPI CX353A network cards I got off ebay, updated their firmware and installed their drivers. After switching transceivers/cable and…
1
vote
0 answers

what is intel omnipath topology error?

I have lustre setup and connected with intel omnipath. the metadata server is getting rebooted often. in logs, i can see the opa topology error. what is this error means?. will it cause the server to reboot automatically ?
1
vote
2 answers

Infiniband drivers : OFED or distro included?

I'm setting up a Linux cluster with infiniband network, and I'm quite a newby in infiniband wolrd, any advice is more than welcome ! We are currently using Mellanox OFED drivers, but our infiniband cards are old and not recognized by the latest…
nirnaeth
  • 23
  • 6
1
vote
1 answer

revive an old QLogic IBA7322 Infiniband card on CentOS8.2

I have a few HP Blades Gen7 equipped with QLogic Infiniband cards IBA7322 which I would like to use with CentOS 8. The problem I have is that I cannot find the right drivers for them. All the information I find is either old, or the links don't work…
lucian
  • 131
  • 1
  • 5
0
votes
1 answer

Infiniband P_Keys and the Linux kernel

I'm trying to configure Infiniband partitions between two Debian based Linux hosts running the 4.15 kernel and A Mellanox SX6036 switch. I've setup a "DMZ" partition on the swich using a PKey of 0x0001 and added the Port GUID numbers from the active…
Server Fault
  • 3,454
  • 7
  • 48
  • 88
0
votes
0 answers

QLogic 12300 :: Can't use more than 6 ports

I have 2 QLogic 12300 InfiniBand switches (12300-BS01) with 36 ports. For some strange reason I am unable to connect anything to ports 7-36, it seems that only ports 1-6 are working. I Googled and found that in 12300 16-port switch (12300-BS16),…
0
votes
1 answer

MPI can not find Infiniband interface point-to-point

I think I have a Infiniband setup issue. I'm not very experienced with setting up Infiniband or networks. If I try to force MPICH to use IB, I get errors: [Bryan@node1 shared]$ ./mpich-3.3.1/bin/mpiexec -hosts=node1,node2 -iface=ib0 -n 4…