Network operators are actively upgrading their IP networks to the next generation of packet interfaces, namely 400 Gigabit Ethernet. As they scale high-speed router connections to enable 400G everywhere, operators need to choose IP-optical solutions that can efficiently support their network use cases.

Building efficient IP-optical networks for the 400G era requires careful consideration of several factors. For example, network operators can take advantage of a range of 400G coherent optic technology options, including pluggable digital coherent optic (DCO) transceivers such as 400ZR/ZR+ or 400G Multihaul, which can be equipped directly in router ports, and high-performance transponder-based optics that enable 400G over any distance.

The best optics technology for IP-optical connectivity depends on the characteristics of the network and application. Are IP-optical 400G connections needed across tens, hundreds or thousands of kilometers? Will the IP-optical connectivity consist of point-to-point links between two locations, or will it be pervasive across much larger and complex network topologies, such as metro rings or regional and long-haul core networks? Are the traffic volumes expected to scale to a few 400G connections, or to multiple terabits of capacity per fiber?

Prior posts in this blog series have examined enabling technologies that can support 400G everywhere, from the latest generation of 400G coherent optics to the features needed in routers and optical line systems.

This post explores some prevalent use cases for network operators that are looking to scale their IP-optical networks to support 400G everywhere. It looks at how operators can apply the different technologies in different architectures to enable the most efficient IP-optical implementations for these use cases and why the best solution may not be the same in all cases.

Point-to-point router interconnection

Some use cases require a simple network topology that consists of a point-to-point link between routers. Examples include access links used to deliver triple-play services to neighborhoods, provide high-speed connections for a business customer, or interconnect data centers across a campus, metropolitan area or region.

Ideally, network operators would deploy pluggable 400G coherent optics into 400GE router ports, selecting optics with enough reach to close the link without the need for intermediate regeneration. Operators can use 400ZR for links up to 120 km, 400ZR+ for links up to 500–750 km and 400G Multihaul for links up to 1,000 km. Transponders equipped with the latest generation of high-performance coherent optics will enable 400G connectivity over links of thousands of kilometers.

For point-to-point links that need to support more than one 400G link, and up to 64 400G connections per fiber in the C-band, operators can use an optical line system to combine multiple 400G wavelengths. For point-to-point applications, operators can deploy simple fixed WDM multiplexers at each end of the link to combine and separate multiple 400G wavelengths., They can use optical amplifiers to boost the 400G signals when the link distance exceeds 40 km.

The following figure shows a point-to-point IP-optical use case that uses pluggable 400G optics in routers for data center interconnection (DCI).

Simple point-to-point applications such as DCI will be early candidates for IP-optical integration using pluggable 400G coherent optics in routers. However, opportunities for IP-optical integration exist across a much broader range of applications in communications service provider (CSP), mobile network operator (MNO) and cable operator networks.

Should the point-to-point model for IP-optical integration simply be duplicated for these other network applications, stitching together a multitude of fixed WDM connections between adjacent routers across the entire network? In practice, pervasively implementing point-to-point connections in applications with hub and spoke traffic demands over rings, or with any-to-any traffic connecting across complex meshed networks, can quickly become inefficient, costly and inflexible. Let’s look at why that is the case.

Metro-regional aggregation and backbone networks

Many network operators aggregate or backhaul traffic from multiple edge nodes to one or several hub nodes, typically in metro areas and sometimes across regional aggregation rings. For example, CSPs aggregate business and residential services from access nodes to a central office, MNOs backhaul traffic from radio sites to packet core locations in the metro, and cable operators distribute triple-play services from a primary head-end node to secondary nodes across a metro area.

The figure below shows an example of edge nodes aggregating to one hub node over a path-diverse fiber ring, with the router connections from the edge to the hub nodes at 400G. Note that the number of nodes in such an application can vary widely, as can the overall circumference of the ring.

With continued scaling of IP bandwidth, traffic volume from each edge node to the hub will efficiently fill one or multiple 400G ports from each edge router. But transiting that traffic across multiple router hops around the ring, simply for it to reach the hub, will be costly and inefficient. The most efficient approach leverages the optical line system to decouple the end-to-end 400G wavelength connections from the physical topology of the ring. This allows direct edge-to-hub connections to bypass intermediate nodes and routers in the wavelength layer.

ROADM-optimized IP topologies

Network operators can create efficient IP topologies through a holistic approach that uses reconfigurable optical add-drop multiplexers (ROADMs) to separate the logical IP topology from the physical fiber topology and allow selective add/drop, or bypass, of 400G wavelengths at any node. This approach can allow operators to, for example, build a cost-efficient hub-and-spoke IP topology on top of a physical ring in the aggregation network.

ROADM-optimized IP topologies use 400G connections to a router where needed, while optically bypassing routers at intermediate nodes when a 400G connection needs to terminate elsewhere. This limits router transits, reduces router port and fabric utilization, lessens coherent optics usage, lowers end-to-end latency, and cuts down on space and power use at each node.

The benefits of ROADM-optimized IP topologies extend to metro and regional IP-optical backbone networks. These networks typically connect many hub nodes in aggregation/backhaul rings to each other, and to internet exchanges, peering points and long-haul points of presence. As a result, metro-regional backbone networks typically have higher traffic volumes than aggregation rings, any-to-any mesh traffic demands, a greater volume of bypass traffic and longer distances between nodes.

Nokia network studies show that when IP traffic scales to a large number of 400G end-to-end packet flows across metro or regional networks, ROADM-optimized IP topologies reduce the number of 400ZR+ pluggable optics required by 65 percent compared to flat, hop-by-hop architectures. These results will be presented at the 2021 Optical Fiber Communications conference, in a session titled “Comparing IP-Optical Architectures & WDM Transport Technologies in Metro, Regional and Long-Haul Networks”.

Similar studies of regional-scale backbone networks show similar results. The difference is that pluggable 400G Multihaul optics outperform 400ZR+ in these larger networks. This is due to the better capacity-reach performance of 400G Multihaul optics, which allow more router ports to operate at 400G rates over more spans and reduce the number of spans that require regeneration.

These results clearly show that using optimized router bypass when IP traffic scales to efficiently fill 400G router ports lowers CAPEX benefits by reducing the number of 400G coherent optics and router ports needed and by limiting router fabric exhaust and router over-builds. This approach also lowers OPEX by reducing the power consumption of optics and routers at intermediate transit nodes.

Routers provide the most value by efficiently aggregating packet traffic at each end of an IP connection, after which that traffic is best transported from end to end in the optical domain. Using ROADMs to allow 400G connections to bypass intermediate routers in the optical domain reduces power consumption per 400G circuit by 97 percent or more at every intermediate node compared to a case where that 400G connection must transit through a router.

Building efficient, application-optimized IP-optical networks

IP networks inherently need to scale and adapt to changing traffic patterns. IP traffic continues to grow, and pluggable WDM coherent optics such as 400ZR/ZR+ and 400G Multihaul will enable operators to scale router connections up to 25.6 Tb/s per fiber across access, metro and regional networks. Operators will complement them with high-performance transponders to enable 400G over long-haul distances.

For simple point-to-point applications such as DCI, operators can pair these 400G coherent optics options with optical line systems to provide WDM muxing and optical amplification that enable efficient scaling over a wide range of link distances.

For metro-regional aggregation and backbone networks, operators can leverage IP-optical integration using 400G pluggable coherent optics with optimized ROADM-enabled optical line systems to:

• Support graceful, in-service evolution of the network as traffic flows and demand patterns change and evolve towards different optimal router topologies
• Efficiently scale end-to-end packet flows while limiting the use of coherent optics and router ports
• Minimize router fabric exhaust and overbuilds by reducing unnecessary transit traffic
• Enable optional optical layer protection to mitigate against fiber and equipment failures

Nokia provides end-to-end IP-optical solutions that enable migration to 400G pluggable coherent optics in routers. Operators can combine these solutions with our full-featured, application-optimized optical line systems to create optimized and efficient IP-optical networks that support 400G everywhere.

Learn more

Technology: Coherent Routing
Technology: PSE Super Coherent Technology
eBook: Beyond the limit: Coherent solutions for the next decade
Technology: FP4 network processor
Solution: IP-optical coordination
White paper: The 400GE inflection point