The Next Station of Data Center Networks—200G vs 400G
Jason Data Center Architect Jan 19, 2023 With the rapid development of mobile internet, cloud computing, big data, and other technologies, the entire communications industry has been demanding high-speed, high-bandwidth, and low-latency networks after the 10G-40G-100G or 10G-25G-100G stages, and urgently needs to upgrade to a higher level. 200G and 400G Ethernet are receiving increasing attention for their applications in data centers, and the choice between 200G and 400G Ethernet has become a hot topic in the industry.
The data center network, as the infrastructure for the survival and development of Internet services, has already moved from the initial Gigabit and 10 Gigabit networks to the stage of large-scale deployment of “25G access + 100G interconnection”.
100G Interconnection: Full-box Architecture Is Valued by Large Internet Companies
Under the architecture of “25G access + 100G interconnection”, the data center network realizes large-scale access through three-level networking, and the scale of a single cluster server can exceed 100,000.
As shown in the figure below, Pods based on T1 and T2 layers can be flexibly expanded like Lego blocks and built on demand.
With the improvement of the capability of large-capacity forwarding chips and the reduction of the cost of 100G optical interconnection, a full-box device networking solution for 100G interconnection with single-chip switch equipment has appeared on the market. This single-chip multi-plane interconnection solution is typically represented by a 12.8T chip. A single chip can provide a port density of 128x100G, and a single POD can provide the access capability of 2,000 servers.
The full-box device networking solution, compared with the traditional frame-box device solution, although the number of network nodes and the number of optical interconnection modules between devices has increased, which has brought about an increase in the workload of operation and maintenance, but due to the introduction of high-performance forwarding chips, It effectively reduces the cost per bit of the data center network port, and is very attractive to large Internet companies.
Network speed up Becomes Inevitable—200G vs 400G
The network has never existed in isolation, and the industrial environment is the big soil that determines whether technology can grow and mature.Let’s first examine the current status of 200G and 400G industries from two aspects: network Protocol standards, and optical transceiver.
Network Protocol standards
200G vs 400G Standards: Protocol Standards Are Mature
During the evolution of IEEE protocol standards, the 200G standard was launched later than the 400G standard.
The IEEE 802.3 Ethernet Working Group (Working Group) established a project to formulate the 400G standard in 2013 after completing the BWA I (Bandwidth Assessment I) project research. In 2015, in order to further expand the market scope to include 50G server and 200G switch specifications, IEEE established the 802.3cd project and started the formulation of 200G standards.
Due to the correlation between 200G and 400G specifications, the 200G single-mode specification was finally included in the 802.3bs project. By then, 400G has basically completed the main design of PCS, PMA, and PMD, and the 200G single-mode specification is generally based on the halving of the 400G single-mode specification.
On December 6, 2017, IEEE 802 finally approved the IEEE 802.3bs 400G Ethernet standard specification, including 400G Ethernet and 200G Ethernet single-mode, and the standard was officially released. IEEE 802.3cd defines the standard for multi-mode beyond 200G and was officially released in December 2018.
As shown in the table below, 400G has achieved standard support for all scenarios, including 100m, 500m, 2km and long-distance 80km.
| Distance | Standard | Name | Rate port | Rate optical port |
|---|---|---|---|---|
| 100 m | IEEE 802.3cd IEEE 802.3cm | 200G SR4 400G SR8 400G SR4.2 | 4 x 56G 8 x 56G 8 x 56G | 4 x 50G 8 x 50G 8 x 50G |
| 500 m | IEEE 802.3bs | 400G DR4 | 8 x 56G | 4 x 110G |
| 2 km | IEEE 802.3bs IEEE 802.3bs 100G Lambda MSA | 200G FR4 400G FR8 400G FR4 | 4 x 56G 8 x 56G 8 x 56G | 4 x 50G 8 x 50G 4 x 100G |
| 10 km 6 km | IEEE 802.3bs 100G Lambda MSA | 400G LR8 400G LR4 | 8 x 56G 8 x 56G | 8 x 50G 4 x 100G |
| 80 km | OIF | 400G ZR | 8 x 56G | DP-16QAM |
Optical modules
200G vs 400G Optical Modules: 400G Is More Cost-Effective and the Industry Is More Mature,we will explore it from below four aspects
Background on 200G and 400G optical modules
A 200G optical module refers to an optical module with a transmission rate of 200Gbps, which is one of the mainstream optical modules currently on the market.
A 400G optical module is a high-speed optical module interface specification that supports high-speed transmission of 400G. It is an upgraded version of the QSFP interface and uses optical fiber as the signal transmission medium. The digital signal is converted into an optical signal and transmitted through the optical fiber.
Differences between 200G and 400G optical modules
The main differences between 200G and 400G optical modules are as follows:
- Different transmission rates: The transmission rate of a 400G optical module is twice that of a 200G optical module. The maximum transmission rate of a 400G optical module is twice that of a 200G optical module.
- Different packaging types: The two main packaging types for 200G optical modules are QSFP-DD and QSFP56. The mainstream packaging type for 400G optical modules is currently QSFP-DD.
- Different costs: The cost of a 400G optical module is higher than that of a 200G optical module because it requires more optical components and more complex manufacturing processes.
Development Trends of 200G/400G Optical Modules
A predictable trend in the data center optical interconnect market is the gradual phasing out of low-speed optical modules for core networks and data centers, with a shift from 10G to 40G and higher, from 40G to 100G. The new development of 100G optical devices has paved the way for 200G/400G optical devices.
Compared to the previous generation of 100G optical modules, 200G optical modules have increased transmission speed by 5 times while significantly reducing costs. Therefore, 200G optical modules have become one of the mainstream optical modules in the market and are widely used in data centers, internet data centers, and enterprise data centers.
400G optical modules can provide higher transmission speeds and larger bandwidth, meeting the demands of higher performance computing, storage, and network devices. Therefore, they are widely used in cloud computing, big data, and artificial intelligence fields. In the future, the technology of 400G optical modules will continue to develop and mature.
Industry Environment Forecast
It can be seen from the below table that when the data center evolves from 10G servers to 25G, and the network interconnection is upgraded from 40G to 100G, the network bandwidth doubles, but the interconnection cost and power consumption remain unchanged, that is, the Gbit interconnection cost and power consumption decrease half. Therefore, 100GE replaces 40GE and becomes the mainstream network interconnection solution in the 25GE era.
200GE and 400GE optical modules are a little different from the past. The traditional optical module adopts NRZ (Non-Return-to-Zero) signal transmission technology, using high and low signal levels to represent 0 and 1 of digital logic signals, and each clock cycle can transmit 1 bit of logic information. Both 200G and 400G optical modules use high-order modulation technology - PAM4 (Pulse Amplitude Modulation 4 fourth-order pulse amplitude modulation). The PAM4 signal uses 4 different signal levels for signal transmission, and each clock cycle can transmit 2 bits of logical information, namely 00, 01, 10, and 11.
Therefore, under the same baud rate conditions, the bit rate of the PAM4 signal is twice that of the NRZ signal, the transmission efficiency is doubled, and the Gbit cost is effectively reduced. From the perspective of optical module composition, both 200G and 400G modules adopt the mainstream architecture of 4-lane, so the module design cost and power consumption are similar.
Because the bandwidth of 400G modules is twice that of 200G, the Gbit cost and power consumption are half of 200G.
On the other hand, in addition to the architecture design, the module cost also depends on the scale of the scale. According to the shipment data of the third-party consulting company Omdia (formerly OVUM), the current layout of TOP8 suppliers in 200G and 400G modules is as follows.
As shown in the figure above, there are only two types of 200G modules, 100m SR4 and 2km FR4, of which there are only two suppliers for 100m SR4. In contrast, there are five types of 400G modules, and TOP8 manufacturers have all laid out 100m, 500m and 2km modules. The industry maturity of 400G is far better than that of 200G, and customers have more choices.
This statistical result also further illustrates that due to the introduction of PAM4 technology, there are technical costs of cost and power consumption. For data center networks that are sensitive to cost and power consumption, the industry is eager to absorb this cost by leapfrogging from 200G to 400G. 400G with the same technology and cost structure is more competitive in terms of evolution.
Summary
The momentum of 400G transmission is obvious, and the 200G generation may be skipped. The data center network exists to serve services. From the perspective of business drivers, the rapid growth of digital construction will promote the rapid increase of 100G servers in 2020 and become the mainstream. From the perspective of cost, since the cost of data center optical devices accounts for more than half of the cost of the entire data center network equipment, due to the introduction of PAM4 technology, the single bit cost of 400G optical devices is more advantageous than that of 200G optical modules, and the deployment cost of optical modules will directly drive the entire network. The overall network construction cost is reduced.
Related Resources:
Typical Solutions for 400G Data Center
Typical Connection Solutions of 400G Optical Transceivers and Cables
400G Optical Module Application Scenarios
400G Solutions Changes High-Bandwidth DCI Architecture
A Beginner Guide: What is 400G Optical Module?