OSFP 800G 2xDR4/400G DR4: A New Choice for InfiniBand Network Architecture

NADDOD Gavin InfiniBand Network Engineer Mar 18, 2024

With the release of large-scale language models such as ChatGPT, the demand for AI computing power has surged, further driving the need for 800G optical modules and accelerating the iteration of high-speed products. The massive training tasks of these models require a substantial cluster of GPU servers to provide computational power, necessitating high-speed optical modules for data exchange between servers.

 

DGX H100 Interconnect

Currently, GPU servers consume significantly more power than traditional data center servers. According to reports, NVIDIA's upcoming AI chip B200 may have a power consumption of up to 1000W. This high power consumption results in a much lower density of GPU servers that can be supported in a single cabinet compared to traditional servers, leading to a decrease in server density. To achieve equivalent computing power, more servers and data centers are required, and for larger-scale clusters, there may even be situations where the distance between servers and switches increases.

 

NVIDlA Data Center Al GPU Roadmap

Meanwhile, despite significant breakthroughs in data transmission rates achieved by high-speed optical modules, they face the challenge of reduced transmission distances. Current 800G optical modules generally use a high baud rate of 53.125G and PAM4 modulation technology to achieve a SerDes rate of 106.25G. However, such high-speed signals are highly sensitive to interference during transmission, inevitably resulting in a reduction in transmission distance while ensuring signal transmission quality. For example, the current transmission distance of 800G OSFP 2xSR4 and 400G OSFP SR4 is only 50 meters, which may suffice for small-scale clusters but may lead to inadequate transmission distances when facing larger-scale clusters.

Finding a balance between high-speed rates and long-distance transmission has become an urgent problem in the industry. To address this challenge, single-mode OSFP 400G DR4 and OSFP 800G 2xDR4 modules have emerged.

 

NADDOD Product Solutions

As a provider of comprehensive optical network connectivity solutions, NADDOD previously introduced OSFP-800G-2xSR4H and OSFP-400G-SR4H optical modules, mainly used in InfiniBand for short-distance transmissions of up to 50 meters. Now, NADDOD introduces OSFP 800G 2xDR4 (500m)/400G DR4 (100m) modules, primarily meeting the demand for longer-distance transmissions under the InfiniBand network architecture.

 

OSFP 800G 2xDR4

 

OSFP 400G DR4

The NADDOD OSFP 800G 2xDR4 optical module features dual-port MPO/MTP-12 connectors, OSFP packaging, and a finned top. It is a high-performance, cost-effective module suitable for InfiniBand NDR and 800G Ethernet applications. This optical module adopts 8-channel 100G-PAM4 signal modulation, 1310nm EML transmitter, and PIN receiver, with a power consumption of less than 16W. With FEC enabled, it can achieve a maximum transmission distance of 500m via OS2 single-mode fiber.

 

OSFP 400G DR4

 

OSFP 400G DR4

The NADDOD OSFP 400G DR4 optical module features a single-port MPO/MTP-12 connector, OSFP packaging, and a flat top. It is a high-performance, cost-effective module suitable for InfiniBand NDR and 400G Ethernet applications. This optical module adopts 4-channel 100G-PAM4 signal modulation, 1310nm EML transmitter, and PIN receiver, with a power consumption of less than 9W. With FEC enabled, it can achieve a maximum transmission distance of 100m via OS2 single-mode fiber.

 

Product Applications

The OSFP 800G 2xDR4 optical module is typically used to connect 2x400G Quantum-2 InfiniBand switches to achieve a speed of 800Gb/s to 800Gb/s, or to interconnect with OSFP 400G DR4 on ConnectX-7/OSFP networks to achieve connections at speeds ranging from 800Gb/s to 2x400Gb/s or 800Gb/s to 4x200Gb/s, as detailed below:

 

800G Switch—800G Switch

 

Each Quantum-2 InfiniBand switch uses a 16W dual-port OSFP 800G 2xDR4 optical module to transmit speeds of 800Gb/s between two switches, as illustrated in the following connection scheme:

 

2x400G Twin NDR Switch Links Using Single-mode Optics

800G Switch—2x400G ConnectX-7/OSFP

 

The dual-port OSFP 800G 2xDR4 optical module supports connection with up to two 400G ConnectX-7/OSFP network cards. Connected via two single-mode fiber patch cords to OSFP 400G DR4 optical modules, it achieves a transmission distance of up to 100 meters.

 

2x400G Twin NDR to 400G HCA Links

800G Switch—4x200G ConnectX-7/OSFP

The dual-port OSFP 800G 2xDR4 optical module can connect with up to four 200G ConnectX-7/OSFP network cards. Connected via a 1:2 branch fiber patch cord to OSFP 400G DR4 optical modules, it can achieve a maximum distance of 100 meters.

 

Note: In this connection method, only two channels of the OSFP 400G DR4 optical module are activated, achieving a 200Gb/s connection rate, while the module automatically reduces power consumption. The power consumption of the dual-port OSFP 800G 2xDR

 

2x400G Twin NDR to Four 200G NDR200 HCA Links

Summary

With the rapid development of artificial intelligence and data center technology, particularly led by the trend of NVIDIA HPC/AI cluster models, there is a growing demand for optical modules and cables to achieve high-speed and low-latency data transmission within SuperPODs. NADDOD provides users with advanced technology-backed OSFP 800G 2xDR4/400G DR4 single-mode optical modules and reliable, efficient solutions, assisting users in overcoming challenges such as decreasing server density and increasing connection distances. In the upcoming digital wave, NADDOD will continue to innovate and develop, committed to providing users with increasingly superior and efficient connectivity experiences.