NADDOD 400G Ethernet Product Connection Solutions

As large-scale data centers transition to faster and more scalable infrastructures, high-capacity connectivity becomes crucial to accommodate the expanding number of users, devices, and applications. Simultaneously, the accelerated adoption of ultra-scale cloud infrastructure and services drives the demand for networks with high throughput and low latency. With 100G networks no longer meeting current application requirements, the next wave of ultra-scale deployments necessitates 400GbE and 200GbE data center architectures to avoid costly fiber optic equipment topology changes. As the next-generation mainstream port technology, 400G can significantly increase network bandwidth, enhance link utilization, and help operators, OTTs, and other customers effectively cope with explosive growth in data traffic.

 

 

Advantages of 400G Deployment

 

Enabling New Scale Horizons: 400G optical transceiver modules offer four times the bandwidth per RU compared to 100G, lowering the cost per bit by providing the same bandwidth in a smaller physical space. The transmission capacity of 400Gb/s per wavelength is also easier to manage, requiring fewer overall ports and reducing power consumption per gigabit, thereby enabling the transmission of more bits at lower power.

 

Trends in 400G Interconnection Technology

 

Since the approval of 400G Ethernet at the end of 2017, technologies such as 100G SerDes, switch chips, and high-speed optical module technologies have matured gradually. The deployment pace of 400G optical network solutions is accelerating, with developments in standardization, commercialization, and scaling. Key product solutions driving the robust development of 400G connection solutions include 400G NICs, 400G switches, 400G optical modules/cables, and crucial technologies like RoCE (RDMA over Converged Ethernet).

 

1. RoCE (RDMA over Converged Ethernet)

 

RoCE, defined in the InfiniBand Trade Association (IBTA) standard, allows RDMA over Ethernet networks. RoCE technology is critical for accelerating data center adoption, as RDMA enables network adapters to transfer data directly to or from application memory without copying data between application memory and the operating system's data buffers. Additionally, RoCE helps ensure low latency, intelligent congestion management, and flexible QoS.

 

2. 400G NICs

 

With increasing network bandwidth demands and server processing capabilities, 400G NICs have become critical components in high-bandwidth-demand data centers. The latest SmartNIC versions offer either two 25G/50G/100G/200G ports or a single 400G OSFP/QSFP112 port, along with 50G PAM4 SerDes capabilities and PCIe 4.0/5.0 x16 host connections. The availability of SmartNICs is crucial for improving network programmability and accelerating application offloading to meet current and future network requirements in high-bandwidth and high-density environments.

 

3. 400G Switches

 

Driven by the construction of data center network capacity for ultra-scale and cloud providers, the global Ethernet switch market is growing rapidly. According to IDC reports, the global switch market growth rate reached 30% in 1Q23, with 200G/400G high-performance switches accounting for 12% of the market share. With the widespread introduction of Ethernet switch chips supporting 100Gb/s per channel rates (e.g., Broadcom's Tomahawk 4-100G), major domestic and foreign manufacturers have successively launched and deployed 400G network devices, such as Cisco Nexus 9000 series, Juniper PTX10000 series/QFX5130/QFX5220, Arista 400G 7060X4 series, NVIDIA SN4700/SN5400, etc. The emergence of 400G data center switches can help data centers upgrade from 100G to 400G, providing flexibility for constructing large-scale leaf-spine designs and reducing the total number of network devices, thereby reducing costs and power consumption. Regarding the port form of switches, from the devices released by major manufacturers, the ports of 400G Ethernet devices are relatively consistent, mainly QSFP-DD type.

 

4. 400G Optical Modules/High-speed Cables

 

High-speed Ethernet optical modules (200G, 400G, and 800G) are the core drivers of the growing demand for Ethernet optical modules. To achieve higher rates, optical modules generally use three methods: increasing the optical device rate (higher baud rate), increasing the number of channels (multi-channel), and using advanced modulation techniques (PAM4) to reduce the transmission cost per bit. The rate growth of optical modules and the iteration of port types are influenced by the demand from the network card and switch equipment sides. Therefore, there are multiple choices for the packaging standards of optical modules: QSFP-DD, OSFP, and QSFP112. Among them, QSFP-DD and QSFP112 can be backward compatible with QSFP56/QSFP28/QSFP+ ports, while OSFP cannot be directly backward compatible due to size differences.

 

QSFP-DD: Quad Small Form Factor Pluggable-Double Density, is an extension of the QSFP interface, which adds a row to the original 4-channel interface, increasing it to 8 channels, hence the term "double density." The number of interfaces is twice that of QSFP28, with the optical side using 850Gb or 4100Gb, and the electrical side consisting of 8 50GE PAM4 physical channels.

 

 

OSFP: Octal Small Form Factor Pluggable. An OSFP module's electrical interface consists of 4 electrical channels, each channel carrying 100Gb/s, providing a total bandwidth of 400Gb/s. The OSFP module interface size is slightly larger compared to QSFP-DD.

 

 

QSFP112: Based on the current QSFP packaging, it supports 4 lanes of 112G-PAM4 electrical and optical signals. Compared to the 8-channel 400GE ports, the physical channel data is further halved to 4 channels.

 

 

Both QSFP-DD and QSFP112 are backward compatible with QSFP56/QSFP28/QSFP+ ports. However, due to size differences, OSFP cannot directly achieve backward compatibility.

 

NADDOD 200G/400G Ethernet Solutions

 

Dual-port 200G QSFP112/QSFP56 NIC + 400G Ethernet Switch Solution

 

QSFP112/QSFP56 NIC - QSFP-DD Switch: It is still recommended to use QSFP56 packaging for modules. Since QSFP112 ports can be backward compatible with QSFP56, when connecting to a switch with a 400G QSFP-DD port, QSFP56 packaging modules or AOC/DAC cables, which are more mature and lower cost, can be used directly.

 

 

For QSFP112 NIC to QSFP112 switch connections: If both the switch and NIC ports are QSFP112, then QSFP56 optical modules or AOC/DAC cables can be used simultaneously.

 

 

Single-port 400G QSFP112/OSFP NIC + 400G Ethernet Switch Solution

 

QSFP112/OSFP NIC - QSFP-DD Switch: 400G NICs commonly come in QSFP112 and OSFP port types. When paired with a QSFP-DD port switch, attention should be paid to the inconsistency in SerDes rates between the NIC side and the switch side (NIC side is 4x100G, switch side is 8x50G). Therefore, when selecting modules on the switch side for connection to the NIC, the choice needs to convert the 8x50G on the electrical side to 4x100G on the optical side. For NIC side module selection, 400G QSFP112/OSFP SR4/DR4 can be chosen, and for switch side module selection, 400G QSFP-DD SR4/DR4 or AOC/DAC cables from QSFP112/OSFP to QSFP-DD.

 

 

For QSFP112 NIC to QSFP112 switch connections: When both ends of the ports are QSFP112, you can choose to use QSFP112 packaged optical modules (400G QSFP112 SR4/DR4) or connect directly using AOC/DAC cables.

 

NADDOD is a leading provider of optical network solutions, focusing on the field of optical connectivity + network products and solutions. With profound business understanding and rich project implementation experience in building high-speed networks and accelerating application deployments, NADDOD's 400GE networking solutions offer various flexible configurations for cloud data centers and AI applications, meeting the continuous growth in data center traffic and the high-bandwidth demands of large-capacity 400G data center network solutions. With highly advantageous and valuable optical network products and comprehensive solutions, NADDOD significantly improves customer business acceleration capabilities.