As hyperscale data centers look for more bandwidth and start to deploy 400 Gigabit Ethernet (GbE) links in their spine networks, there are already moves towards a new 800 GbE standard. The industry is already deploying 2×400 GbE (an aggregate of 800G) in QSFP-DD800 and OSFP form-factors in support of 25.6 Tb switches and line cards. The Ethernet Technology Consortium (ETC) has defined an 800 GbE standard based on dual instance of 400 GbE PCS/FEC. In addition, 800G pluggable transceiver modules have been specified by the 800G Pluggable MSA, the QSFP-DD MSA and the OSFP MSA.

Coherent 400ZR optics is a key driver of 400 GbE in data centers by interconnecting regional data centers using DCI (Data Center Interconnect) by transporting Ethernet directly on coherent links. Currently there are some proprietary 800G per wavelength coherent optic transceivers already available. However, for interoperability, the Optical Internetworking Forum (OIF) is now working to define 800ZR to address DCI applications and 800G-LR to address single wavelength coherent up to 10 km for campus applications.

So, what is the status of 800G standard development? Currently, the IEEE 802.3 has formed the 802.3df task force to specify 800 GbE and 1600 GbE. The IEEE 802.3df task force development can be categorized in to three main projects:

1. Define 800/1600 GbE MAC, PCS, and FEC. Define PMDs in support of 800 GbE, such as 800-KR/CR, 800G-SR8 (100 m), 800G-DR8 (500 m), 800G-FR8 (2 km), 800G-LR/LR8 (10 km), and 800G-ER/ER8 (40 km).
2. A major effort will be to define new 200G per lane optical PMDs. Define PMDs such as 800G-DR4 (500 m), 800G-FR4 (2 km), and 800G-LR4 (10 km).
3. A major effort will be to define 200G per lane electrical IOs. Define PMD such as 200G-KR/CR, 1600G-DR8 (500 m), 1600G-FR8 (2 km), 800G-LR (10 km), and 800G-ZR (80 km).

IEEE 800G standards cover 8x100G using multimode fiber for up to 100 m, 8x100G parallel single-mode fiber for up to 500 m, and 8x100G wavelength division multiplexing (WDM) over single-mode fiber for up to 2 km and 10 km. Followed by more efficient PMDs based on 200G per lane. IEEE 802.3df needs several years to develop new Ethernet standards beyond 400G, so the Ethernet Technology Consortium stepped in to define an 800 GbE MAC/PCS standard based closely on the existing 400G Ethernet. By simply doubling the number of FEC/PCS VL (virtual) lanes (from 16 to 32), a new media access control (MAC) has been specified that is eight by 106.25 Gbps and operates at 800 GbE. By mostly leveraging the 400G IEEE 802.3bs standard, the Ethernet Technology Consortium saved considerable time and effort in developing the new 800G specification that they are calling 800G-ETC-R (to differentiate it from any future IEEE standard).

The following figure shows the high-level architecture of an Ethernet Technology Consortium 800G interface. The MAC has been scaled up to 800G, but there are two physical coding sublayers with forward error correction (FEC) from the 400G Ethernet standard that have only been slightly modified. The MAC distributes data across 32 lanes to the two PCSs, 16 lanes of 25 Gbps data to each PCS. Modified alignment markers are inserted into each PCS data lane to ensure that an 800G data stream can be received and processed. The 32 PCS lanes are then multiplexed in the Physical Medium Attachment (PMA) layer to feed eight 106.25 Gbps lanes to the Physical Medium Dependent (PMD) layer.

The challenge the industry is now facing is to adopt ETC 800 GbE PCS/FEC, which has 32 VLs, or define a more efficient PCS/FEC in the IEEE 802.3df based on 8 VLs for 800 GbE and 16 VLs for 1.6 TbE.

The IEEE 802.3df task is a major undertaking by defining both 800 GbE and 1.6 TbE, which includes defining 13 optical PMDs, 6 copper (Cu) PMDs, and 6 AUIs. All the PMDs based on 100G per lane will be in the fast track project (~2023), the 200G per lane optics in the follow on project (~2024), and the 200G per lane Cu/CR will be in a later project (~2026).

A key part of 800G adoption in data centers depends on the availability of 800G pluggable optical transceiver modules. The 800G Pluggable MSA group have specified a low-cost 8x100G module and a 4x200G module. The low-cost 8x100G module specifies an optical interface consisting of eight parallel single mode (PSM) fibers for lengths from 2 meters to 100 meters, but the IEEE 802.3df expect to define a similar PMD with a reach of 500 meters. The 4x200G module specifies an optical interface that uses wavelength division multiplexing (WDM) to transmit 200 Gbps signals over four different wavelengths on one single mode fiber of length 2 meters to 2 km. Transceiver form factors and electrical interfaces are not specified by the 800G Pluggable MSA, although both QSFP-DD800 and OSFP have defined 800G interfaces.

In addition, the QSFP-DD MSA has defined an 800G module standard called QSFP-DD800 that supports an 8x100G interface. This module specification includes eight electrical lanes as well as the cage and connector system. The QSFP-DD800 modules are also backward compatible with 8-lane QSFP-DD and 4-lane QSFP28.

The 800G Pluggable MSA specification for the 4x200G WDM module is in anticipation of 200G per lane becoming the future preferred choice for Ethernet speeds of 800G and above. This standard also proposes the inclusion of an additional FEC implemented in the module to ensure reliable operation. Therefore, the current focus for 800G is an 8x100G solution with the 4-lane solution being something that will be more cost-effective in the future.

With the 800G-ETC-R standard and 8x100G pluggable module solutions being the current focus for 800G Ethernet, there remains the issue of cabling and connectors within the data center. The 8x100G solution requires 16 fibers for each link. Multi-fiber Push On (MPO) connectors and cables include a number of options that accommodate 16 fibers. MPO-16 and MPO-12 Two Row (also called MPO-24) are obvious choices. A dual MPO-12 connector is a new addition that can be used for 800G. As well as these MPO connectors, there are new very small form factor connectors such as SN and MDC that support flexible port breakout options. As 25.6 Tbps switches start to be deployed in data centers, the 800G port breakout enables up to 256 ports of 100G connections from one switch. These high-radix switches offer higher density and better efficiency in data centers. This trend will only continue when 51.2 Tbps switching silicon becomes available and ports supporting 800G and possibly 1.6T start to appear.

Another step towards 800G Ethernet is being taken using coherent optics. Coherent optics are typically used for data center interconnect applications where it can transport more data over a single fiber for greater distances using higher order modulation techniques, resulting in better spectral efficiency with lower power consumption.

The Optical Internetworking Forum (OIF) has successfully developed 400ZR, a 400G coherent optical implementation agreement that supports links from 80 km up to 120 km. The standard uses dual-polarization DP-16QAM modulation at a 59.84 Gbaud symbol rate with a tunable laser for dense wavelength division multiplexing (DWDM) systems. The interoperable 400ZR pluggable optics make use of low-power digital signal processors (DSPs) that are embedded in the module and compensate for signal dispersion in the fiber.

Now the OIF is in the process of leveraging the 400ZR standard to define 800G coherent optical standards for 2-10 km (800LR) and 80-120 km (800ZR). The 800ZR standard is targeted at data center interconnect applications and includes a tunable laser for amplified DWDM links, whereas the lower cost 800LR is a fixed-wavelength standard more suited for campus use cases. The development of these standards will entail significant technical challenges. To reach 800G transmission over a single wavelength, higher order modulations (32QAM and 64QAM) and higher symbol rates of 90 Gbaud or more need to be considered.

With hyperscale data centers continuously struggling to keep ahead of surging volumes of data traffic, 800G Ethernet switches and pluggable optical modules will provide some relief. The Ethernet Technology Consortium’s 800G standard will help deliver 800G Ethernet solutions until IEEE standards are completed some years later. As 800G Ethernet switches start to hit the market, 8x100G pluggable optics should also be available in QSFP-DD800 and OSFP form factors. Looking further ahead, 200G-per-lane signaling will provide an added boost to 800G and lay the groundwork for 1.6Tb Ethernet.

As AI gains importance and is applied to more applications, the demand for AI accelerator hardware will continue to grow. Whether it is AI training in a data center or AI inference at the network edge, AI hardware will keep evolving, increasing in both processing power and efficiency. The OCP initiative to specify an open modular AI infrastructure that supports interoperability and flexibility has undoubtedly helped AI solution providers in their development efforts. As new AI accelerator hardware emerges, it can now be quickly deployed to power through the data of many new complex AI applications.