ROADMs for the 400G-era of IP-optical networking
Pluggable 400GE digital coherent optics (DCOs) such as 400ZR/ZR+ and 400G Multihaul enable routers to directly integrate IP traffic into 400G wavelengths. For network operators, the question is how to use this technology to minimize overall IP-optical network cost. A key element in achieving this goal is to implement optimized, robust and flexible IP topologies. This blog explores how reconfigurable optical add/drop multiplexers, or ROADMs, can play an important role in enabling efficient IP-optical networks.
Optimizing IP architectures
To control end-to-end cost, network architects strive to minimize the number of hops required to transport traffic between its source and destination. After all, each intermediate node adds latency and consumes valuable resources such as fabric capacity, interface ports, pluggable optics, space and power to pass traffic along the data path.
These incremental transport costs should only be incurred when they are offset by statistical multiplexing gains of aggregating ingress packet traffic along the data path. This happens, for example, when low-capacity (e.g., 10GE and 40GE) access ports are concentrated onto high-capacity 100GE and 400GE egress ports. This IP aggregation is essential for optimizing available port and link capacity, while addressing requirements such as traffic burstiness, busy hours, failures, redundancy and load balancing.
However, the incremental gains of statistical multiplexing have diminishing returns. Once enough IP traffic has been aggregated to efficiently fill a 400GE port, there are little additional packet aggregation gains from subsequent router hops. A cost-optimized 400GE IP transport topology should strive to provide direct, single-hop 400 Gb/s IP transit links between source and destination routers, that minimizes intermediate router hops.
Factoring in fiber topology
In an ideal world, 400GE-capable routers would be directly interconnected over dedicated point-to-point fiber links. In real-world networks, however, the shortest path between two points is often neither straight nor short. For example, fiber in metro-regional access and aggregation networks is typically laid as interconnected and overlapping rings that can span tens to hundreds of kilometers, and interconnect multiple central offices and internet exchange locations. Additional fiber for dedicated router-to-router express paths may also not be available everywhere it is needed. Trenching fiber is costly and time consuming, and may not always provide a direct connection between any two points.
This is where the “optical” in IP-optical integration comes in. Using a combination of wavelength-division multiplexing (WDM) for fiber gain and optical line systems enabled with ROADMs allows network operators to create virtual fiber paths between any two points over complex fiber topologies. These virtual paths allow IP routers to share a common fiber network, such as a ring or interconnected fiber mesh topology, while simultaneously enabling single-hop leaf-spine IP topologies over dedicated wavelength connections.
By selectively adding or dropping wavelengths at any intermediate node, ROADMs can drop router ports to a locally connected router when needed, or pass them to a remote router through the optical layer. In essence, ROADMs virtualize the physical network by introducing a logical transport underlay of point-to-point single-hop router connections that can then be dynamically provisioned and optimized to match the IP traffic topology (Figure 1).
Figure 1. Virtualizing the fiber network with ROADMs
In contrast, connecting end-to-end packet flows over point-to-point WDM links results in multi-hop router connections. This, in turn, leads to significant transit traffic that will have to traverse many intermediate router hops to get to the hub router, consuming most of the router fabric capacity along the way.
Because IP transit traffic can no longer bypass intermediate routers, the cost efficiency of this IP transport model depends on the number of router hops that IP traffic must traverse to reach the final destination. This is a non-issue for simple single-hop, point-to-point applications such as data center interconnect. But for network applications such as access and metro aggregation rings and metro-regional core networks, it can result in inefficient and inflexible IP network topologies.
Port cost, capacity and reach
Saving cost is a lot like trying to lose weight. There are no quick wins, shortcuts or magic pills. It requires discipline and determination to trim costs and nurture a leaner, healthier and more productive network. All too often, savings turn out to be deferred costs or temporary gains. To achieve real and recurring cost savings, operators need an end-to-end network perspective and a vision on how to build smarter and more efficient networks that maximize the cost synergies between IP routing and optical transport without trading off value elsewhere.
One key variable that impacts the cost of ROADM-enabled optimized single-hop IP architectures is the reach of the WDM optics used to interconnect routers. Router-pluggable 400G DCOs such as 400ZR+ and 400G Multihaul enable cost, power and space savings compared to the use of off-board DWDM transponders. They also eliminate the need for gray router optics to connect with the transponder. However, the capacity–reach and number of ROADM pass-throughs supported by router-pluggable DCOs can be limited. Operators must assess the use case for each type of optic against the distance and topology attributes of the desired end-to-end IP connection.
With respect to capacity–reach performance, 400ZR+ DCO is analogous to a short-haul airplane with the capacity to fly up to 500 km with 400 passengers or longer distances with fewer passengers. 400G Multihaul extends the analogy by providing planes with longer ranges over 400ZR+ and through a greater number of cascaded ROADMs. DWDM transponders are cost-optimized to carry even greater payloads over long-haul fiber spans and through even more ROADMs.
Network links have different rate and reach requirements. To optimize link costs, network operators need a range of optics with capacity–reach specifications suited to different applications and use cases. Each type of coherent optic has its own capacity–reach sweet spot. Operators should target the use of optics such as 400ZR+, 400G Multihaul and 400G long-reach transponders for the network applications where they can provide the most cost-optimal IP transport solution.
Table 1. Optimizing capacity, reach and link cost
Forwarding traffic at the most economical layer
Network transport economics depend on the capacity and reach requirements of end-to-end IP traffic flows. The goal is to forward that traffic at the most economical layer. When bursty packet traffic requires aggregation, this is best done by leveraging the scale and service management features of routers in the IP layer. Once this IP traffic has been efficiently aggregated to router ports and allocated to end-to-end wavelength paths (IP over DWDM), the most cost- and power-efficient way to route this traffic through intermediate nodes is via the optical layer using ROADMs.
Figure 2 compares the relative cost and power consumption of routing 400GE flows as either packets, or wavelengths through intermediate router hops. Unnecessary router transits incur an optical–electrical conversion that consumes at least two 400GE ports equipped with DCOs, 400G in IP fabric capacity, and at least 100 watts in power and cooling. In contrast, routing a 400 Gb/s flow as a wavelength using a ROADM-enabled optical line system consumes approximately 4 watts per 400G hop, saving considerable CAPEX and OPEX. Operators can also use the optical network layer to provide an efficient additional layer of protection against fiber cuts for IP traffic.
Figure 2. Power utilization per 400GE packet flow at intermediate node for router transit versus wavelength bypass
Because the majority of IP traffic flows to a fewer large destinations such as hub routers, peering points and data center gateways, the physical transport layer can be engineered to effectively support the main IP traffic routes with direct point-to-point wavelengths or fiber spans. Source-based routing techniques such as segment routing can then efficiently steer traffic to its destination along the shortest and best network itinerary with the smallest number of IP routers and ROADM hops required.
ROADMs to enable 400G everywhere
Nokia offers a new generation of compact and modular line systems with ROADM capabilities, along with a full range of coherent optics options that include router-pluggable DCOs and high-performance transponders. Our comprehensive 400G solution provides the capacity-reach, efficiency and flexibility that network operators need to cost-effectively implement optimal IP topologies for any application and any topology.
