What is 400G Ethernet？

With the adoption of technologies such as artificial intelligence, open systems, machine learning, and cloud storage, enterprises are experiencing an increasing demand for bandwidth and data transmission in their network infrastructure. Unfortunately, 100G and 200G Ethernet technologies are inadequate to meet these requirements.

 

Currently, these technologies are being replaced by 400G, which is renowned for its high data transfer speeds and ability to fulfill complex bandwidth demands. In 2020, various network prognostications and cloud service providers (CSPs) reported that the deployment of 400G would outpace that of 100G and 200G. So, what is 400G Ethernet? NADDOD will now provide a detailed explanation.

1. 400G Ethernet Overview

In recent years, Terabit Ethernet (TbE) has gained favor among network engineers. This trillion-bit Ethernet refers to Ethernet transmissions conducted at a speed of one trillion bits per second. It signifies Ethernet speeds surpassing 100 Gbps.

 

IEEE 802.3b defines 400 Gigabit Ethernet (400BASE) and 200 Gigabit Ethernet (200BASE). The 400 Gigabit Ethernet, referred to as 400G, is four times faster than 100G and represents the most advanced form of cloud infrastructure. This 400G Ethernet is also known as 400GbE and 400GB/s in the data center environment. These terms are often used interchangeably. The following points will help you better understand it:

 

400G: In the data center environment, it is common for network professionals to use 400G Ethernet switches or 400G Ethernet devices. This means that the switches or devices have an Ethernet interface capable of carrying 400G wavelengths.

 

400GbE: This represents the advanced speed of Ethernet interfaces. Switches or any hardware described as 400G must meet the parameters for data management, physical layer, and media access control (MAC) mentioned in the 802.3bs standard.

 

400GB/s: This indicates that at any given time, 400 billion bits of data will be transmitted through a single optical wavelength.

2. Impact of 400G Ethernet on Enterprise Networks

400G Ethernet enables enterprises to plan for expansion, which means catering to high traffic demands. The following points will help us better understand it:

 

The adoption of 5G is driven by the increasing demand for bandwidth and growing traffic. The adoption of 5G implies a significant increase in bandwidth. 400G enables enterprises to engage in small cell development, which is a requirement for 5G deployments.

 

High-performance computing has become prevalent across various industries. Retail, healthcare, oil and gas, and weather forecasting are all experiencing its impact. Real-time data analytics are key growth drivers in these industries. This will accelerate the adoption of 400G Ethernet. The combination of 400G and high-performance computing will help optimize operational and financial efficiency.

 

During the COVID-19 pandemic, there has been a surge in demand for hyperscale data centers, or large-scale data centers, to handle massive amounts of data. As e-commerce and online sales reach their peak, enterprises face the challenge of investing in infrastructure that can support growing data processing and storage needs. Enterprises investing in these hyperscale data centers are considering 400G Ethernet as it can effortlessly meet complex data processing requirements.

3. Deployment Challenges of 400G Ethernet

While 400G is considered essential for growing enterprises, it also faces deployment challenges.

 

400G is standardized by IEEE; however, compatibility of 400G-compliant devices varies across brands. They need to address numerous details such as design, manufacturing, specifications, and more. Despite various tests for device interoperability conducted globally showing positive results, real-world validation is still required. This will help us understand the performance of this technology in various enterprise networks and identify implementation bottlenecks.

 

The 400G Ethernet infrastructure utilizes QSFP transceivers typically designed for 40GbE. This introduces heat-related issues that can result in optical distortions, packet loss, and fiber alignment problems. Similar challenges are encountered when using new modules like CFP8 and QSFP-DD. These transceivers need testing in demanding environments before being deployed in 400G infrastructure.

 

400G Ethernet adopts four-level pulse amplitude modulation (PAM-4). However, early 100G Ethernet used non-return to zero (NRZ) modulation. Decoding and encoding these four levels require the highest level of complexity, and PAM-4 is sensitive to noise. Manufacturers, in general, have found PAM-4 optical signaling too complex and challenging to achieve, making interoperability between devices difficult to achieve.

4. 400G Ethernet Deployment Trends

• QSFP-DD

QSFP-DD technology is addressing the industry's need for improved high-density networking solutions in the face of rapid growth. QSFP-DD represents the first time that a universal form factor is available across the entire range, from copper cables to long-reach optical devices. The QSFP-DD Multi-Source Agreement (MSA) group has also released the Common Management Interface Specification (CMIS) 4.0 revision for the QSFP-DD form factor.

 

The development of CMIS 4.0 aims to enable the use of a common codebase across products and modules while addressing the shortcomings of previous versions of pluggable optical management systems. CMIS also helps ensure backward compatibility of QSFP-DD modules with all QSFP-based transceivers, including widely deployed 40G/100G/200G products.

 

With the growth in adoption of 400G, QSFP-DD provides the most compact 400G modules and the highest port bandwidth density for the data center market. Through rigorous industry testing, the functionality of QSFP-DD has been validated as the ideal high-density, high-speed pluggable module for 400G data center network environments.

 

• Remote Direct Memory Access (RDMA) based on Converged Ethernet (RoCE)

The 400G data network is critical for accelerating the adoption of RoCE technology in data centers. RDMA enables zero-copy networking, allowing data to be transferred directly to or from application memory without the need to copy data between the application memory and the operating system's data buffers. The RoCEv2 protocol also supports RDMA across L3 boundaries.

 

Due to the RoCE architecture, data center network switches achieve performance improvements by enabling explicit congestion notification (ECN) directly, providing line-rate throughput. Additionally, it helps ensure low latency, intelligent congestion management, and QoS flexibility.

 

• Smart Network Interface Cards (Smart NICs)

SmartNICs play a key role in enabling 400G to meet the new high bandwidth demands of data centers. The latest SmartNICs currently available provide two 25G/50G/100G ports or one 200G/400G port, along with 50G PAM4 SerDes capabilities and PCIe 4.0 host connectivity. The availability of SmartNICs is an integral part of improving network programmability and accelerating application offloading, which is crucial for enhancing data center efficiency.

5. Where to Find the 400G Product Manufacturers

Cisco

Cisco's Nexus 9000 series products support comprehensive 400G solutions. Cisco has announced that its Nexus series products related to 400G will meet the requirements of all network architectures and provide high-density 100G/400G support cost-effectively. The entire system will have greater flexibility.

NVIDIA

In April 2022, NVIDIA released the next-generation Ethernet platform, NVIDIA Spectrum-4. The platform consists of NVIDIA Spectrum-4 switches, ConnectX-7 SmartNICs, NVIDIA BlueField-3 DPUs, and DOCA data center infrastructure software, enabling significant acceleration of large-scale cloud-native applications.

 

Spectrum-4 claims to be the world's first 400Gbps end-to-end network platform, with a single-chip switching throughput of 51.2Tbps, four times higher than the previous generation, providing high network performance and robust security for large-scale data center infrastructures. The SN5000 switches powered by Spectrum-4 can support up to 128 400GbE ports or 64 800GbE ports. Spectrum-4 also optimizes RoCE network architecture and provides Adaptive Routing and enhanced congestion control. It offers encryption bandwidth of 12.8Tbps, supporting hardware-level MACsec and VXLANsec.

NADDOD

Since 2017, NADDOD has been continuously providing innovative, efficient, and reliable products, solutions, and services to users, offering optimal switches and connectivity accessories for data centers, high-performance computing, edge computing, artificial intelligence, and other application scenarios. Their product range covers AOC/DAC/optical modules with Ethernet rates of 400G, 200G, 100G, and 56G, significantly improving customers' business acceleration capabilities with low cost and outstanding performance.

 

NADDOD focuses on customer-centricity and continuously creates excellent value for customers from various industries. With a professional technical team and rich experience in implementing and servicing application scenarios, their high-quality products and solutions have gained trust and popularity among customers. They are widely applied in industries and key sectors such as high-performance computing, data centers, education and research, bio-medicine, finance, energy, autonomous driving, internet, manufacturing, and telecommunications.

 

The era of 400G is approaching, and choosing NADDOD as a partner for 400G network deployment enables businesses to obtain flexible, reliable technical support, and product solutions, facilitating a smooth transition during enterprise network upgrades.