Introduction of InfiniBand 100G EDR Solution - NADDOD Blog

Introduction of InfiniBand 100G EDR Solution

NADDOD Abel InfiniBand Expert May 16, 2023

Complex workloads require ultra-high-speed processing of high-resolution simulations, extreme-sized data sets, and highly parallelized algorithms. As these computing demands continue to grow, InfiniBand provides the huge leaps in performance needed for high-performance computing (HPC), artificial intelligence, and hyperscale cloud infrastructures, delivering unparalleled performance at lower cost and complexity. Deploying 100G QSFP28 EDR transceivers and cables over InfiniBand protocol links can improve network latency and enable higher data rates, so this article will give you a brief overview of the NADDOD 100G EDR InfiniBand solution.

100G EDR InfiniBand

1. Introduction of InfiniBand HDR 100Gb/s Cables and Transceivers

InfiniBand HDR 100Gb/s Active AOC Cables

QSFP-100G-AxH is a QSFP28 VCSEL-based (Vertical Cavity Surface Emitting Laser) active optical cable designed for use in InfiniBand 100Gb/s EDR systems and is available in 1m, 3m, 5m, 7m, 10m, 15m, 20m, 30m. 100G EDR active fiber cables are the most popular in high-speed InfiniBand high-performance computing (HPC) environments because they offer predictable latency, very low power (2.2W), and the ability to increase airflow. Because AOC is hot-swappable, it is easy to install and replace. QSFP-100G-AxH contains four multimode fiber (MMF) transceivers per end, each with a data transmission rates of up to 26 Gb/s. They provide more airflow, tighter bend radius and longer transmission distances than DAC cables.

EDR AOC

InfiniBand HDR 100Gb/s Passive Copper Cables

QSFP-100G-CUxH is a high-speed, cost-effective fiber cable alternative for InfiniBand 100Gb/s EDR applications, available in 1m, 2m, 3m, and 5m. 100Gb/s QSFP28 passive copper cable contains four high-speed copper pairs, each with data rates up to 25Gb/s. Each QSFP28 port includes an EEPROM providing product information that can be read by the host system. can be read by the host system. The passive DAC solution provides efficient connectivity for short-haul interconnections. It enables higher port bandwidth, density, and performance. It enables higher port bandwidth, density and configurability at a lower cost and reduces power demand in the data center and reduces power demand in the data center.

EDR DAC

InfiniBand HDR 100Gb/s Optical Modules

The 100Gb EDR optical module adopts QSFP28 package, with LC duplex or MTP/MPO-12 connector, hot-swappable, suitable for EDR InfiniBand, also can be used in 100G Ethernet. It adopts four high-speed differential signal channels, each at 25gbps, and can be divided into QSFP28 SR4, QSFP28 PSM4, QSFP28 CWDM4, and QSFP28 LR4 according to the transmission distance.

100G QSFP28 Cable

QSFP28 SR4 PSM4 CWDM4 LR4
Interface Type MPO/MTP-12 MPO/MTP-12 LC Duplex LC Duplex
Rate EDR EDR EDR EDR
Fiber Type MMF SMF SMF SMF
Wavelength 850nm 1310nm 1271nm,1291nm,1311nm,1331nm 1295.56nm,1300.05nm,1304.58nm,1309.14nm
Transmission Distance 70/100m 2km 2km 10km
Laser Type VCSEL DFB DML DML
DDM Support Support Support Support
Protocol InfiniBand EDR InfiniBand EDR InfiniBand EDR InfiniBand EDR

2. 100Gb InifiniBand EDR System Scenario Applications

InfiniBand is now widely used in data centers, artificial intelligence, AI and other fields, and InfiniBand mainly adopts the spine-leaf architecture. Taking data centers as an example, transceivers and cables are mainly used for.

(1) Data center to user, generated by accessing the cloud for end-user behavior such as browsing the web, sending and receiving emails and video streaming; i.e., switches are interconnected with servers or storage devices.

(2) Data center interconnects, primarily for data replication, software and system upgrades; i.e., switch to switch interconnects.

  • Switch to switch interconnects
    Spine and Leaf configurations of HDR100 InfiniBand b
    uilt at 100 Gb/s, also connects Core Switch and Spine Switch at 100G EDR rate.

    EDR connection
Number Product Category Distance Application Area
1 QSFP-100G-CUxxxH DAC ≤3m Spine switch to Leaf switch
2 QSFP-100G-AxxxH AOC 3≤distance≤30m Spine switch to Leaf switch
3 QSFP-100G-SR4H Optical Transceiver 30≤distance≤100m Spine switch to Leaf switch
4 QSFP-100G-PSM4H Optical Transceiver ≤2km Spine to Core switch
5 QSFP-100G-CWDM4H Optical Transceiver ≤2km Spine to Core switch
6 QSFP-100G-LR4H Optical Transceiver ≤2km Spine to Core switch
  • Switches to servers or GPU computing systems
    DAC is widely used in data centers because it contains no electronics and has the lowest latency. And AOC active optical cable is a highly efficient integrated cable assembly product designed for short distance multi-channel data communication and interconnection applications, communication distance from 1m to 100m, suitable for supercomputers with InfiniBand standard based SDR, DDR and QDR, etc. It has better air flow heat dissipation in the server room cabling system, and the bend radius of optical cable is made smaller than copper cable, and the transmission distance is also better. smaller, the transmission distance is farther, and the BER of the product transmission performance is also better. However, when the transmission distance is farther, optical modules need to be considered.

    Highway
Number Product Category Distance Application Area
1 QSFP-100G-CUxxxH DAC ≤3m Leaf to NIC/Storage/td>
2 QSFP-100G-AxxxH AOC 3≤distance≤30m Leaf to NIC/Storage
3 QSFP-100G-SR4H Optical Transceiver 30≤distance≤100m Leaf to NIC/Storage

3. Summary

With the development of 100G InfiniBand EDR becoming more and more mature, NADDOD 100G EDR InfiniBand solution with high bandwidth, low latency, low power consumption, can achieve higher port bandwidth, density and configurability at a lower cost, and reduce the power demand of data centers, which is ideal for large-scale data centers, high-performance computing, artificial intelligence and future network expansion. It is ideal for large-scale data centers, high-performance computing, artificial intelligence, and future network expansion; customers can choose the corresponding products based on application requirements, transmission distance, network deployment, etc.