The Not-So-Secret Ingredients for Building a Utilities Packet-Optical Network
What are the top considerations when it comes to utilities building a packet-optical network? Ciena's Mitch SImcoe serves up the top ten.
Utilities are implementing highly intelligent energy grids to improve operating efficiency, address consumer demands, and meet government mandates. These Smart Grids are powered by a two-way communications network that must be highly reliable and low latency, yet affordable to install and operate. Once utilities have made the decision to pursue modernization of their grid networks from TDM to packet-optical, there are many criteria to consider. As you are gathering information around your requirements, here are ten ingredients you’ll want to make sure are included in your RFI/RFQ specifications.
- Security and Encryption
Cybersecurity was ranked as the #2 priority for utilities in North America according to a recent survey by consultant Black & Veatch. So, any requirements for security should be at the top of any RFQ issued.
One area of security is related to in-flight encryption. While optical networks are perceived to be very secure, did you know that a $200 coupler purchased off the internet can tap into a fiber without detection allowing a hacker to access the data stream? The only way to protect against this is via Layer 1 in-flight encryption at the 10G/100G and even 200G rates. Encryption at Layer 3 and higher typically adds lots of overhead and latency while Layer 1 encryption performed in hardware adds little overhead or latency. This is key for utilities. Ensure that in-flight encryption cannot be turned off and that the encryption solution has been FIPS 140-2 certified, making it suitable even for meeting the requirements of government defense agencies. - Optical Reach
With the move to renewables and hydroelectric power, the sources of power generation may be hundreds of km from where energy users are located over remote terrain. A couple of examples of this include hydro power in Northern Canada being carried over several hundred km of remote power lines to users in Southern Canada and wind power in Oklahoma being transmitted over a 1000km of high voltage lines to users in the Southeast United States. Some of these networks pass over isolated territories with limited access to communications facilities. When building a packet-optical network to address these challenges, utilities need to consider the need to transport 40/100G payloads up to 300km without regenerators to minimize the number of remote utility facilities. Stable optical performance over these long distances is also key to success. - Packet/Optical Flexibility
Utilities are facing a transition from TDM (SONET/SDH) networks to packet-optical. Yet in many cases, utilities desire the flexibility to maintain some parts of their TDM network over a packet-optical backbone. OTN is one protocol that maintains existing SONET/SDH payloads and performance while allowing the network to scale over a modern packet optical infrastructure. To allow both TDM and packet to co-exist, utilities should demand that their vendors deliver a centralized OTN/Packet fabric allowing for full utilization of each wavelength, any-to-any connectivity, and the ability to drop 1G or 10G at each substation. This provides utilities with full flexibility to either maintain their legacy networks or slowly migrate to packet at the rate that best meets their operational needs.To allow both TDM and packet to co-exist, utilities should demand that their vendors deliver a centralized OTN/Packet fabric allowing for full utilization of each wavelength, any-to-any connectivity, and the ability to drop 1G or 10G at each substation.
- Ultra-Low Latency Performance
Utilities have built their existing networks with the low latency and deterministic performance of SONET/SDH. As utilities move to a packet-optical architecture, the goal is to preserve the performance characteristics of TDM, which are presently available in SONET/SDH, with no performance degradation when transported over Carrier Ethernet as a WAN transport protocol. Utilities should specify that their vendors meet the performance requirements specified in IEEE 1646 and IEC TR 61850-90-12 as well as relay manufacturer requirements for asymmetry and restoration. This requires vendors to deliver an end-to-end substation to control center solution with less than 5 ms round trip latency. - Protection Failover: Core
As another key requirement for performance, Utilities demand equal or better protection switch times to SONET/SDH in case of a network failure in order to keep the lights on. Utilities should demand of their vendors switching performance in Core WAN rings of less than 20 ms to meet this level of performance. - Protection Failover: Edge
Much as we considered network healing performance in the core, Utilities need to consider protection switch times at the edge. Network healing performance for utility service paths can be optimized by provisioning multiple unprotected point-to-point tunnels through the core Carrier Ethernet network. Network healing is then performed by the OT edge device rather than by the core Carrier Ethernet network. Test results show that a significant performance advantage can be achieved by using the OT edge network to perform network healing. Using this approach, utilities should specify that protection switch times at the edge do not exceed 5 ms.Test results show that a significant performance advantage can be achieved by using the OT edge network to perform network healing. Using this approach, utilities should specify that protection switch times at the edge do not exceed 5 ms.
- Multi-Vendor SDN
Utilities have deployed a multitude of vendors across their networks over the years. As staff of legacy technologies are retiring, and in order to lower OPEX, utilities should be looking for vendors that can deliver end-to-end service orchestration and network analytics across multiple technologies and vendors. This includes modern packet networking as well as the utility’s existing Core Routers and legacy TDM. Utilities should specify that vendors deliver solutions that can work across their own network elements as well as other vendors that already may be in their network that can be managed through a “single pane of glass.” - Multi-Vendor Analytics & Topology Discovery
Advanced analytics refers to sophisticated and modern techniques that uncover deep, meaningful patterns in data or content that help utilities make intelligent decisions for meeting specific business and/or operational goals. The network—over which terabytes of traffic traverse every day—serves as the greatest source of this valuable information. With predictive analytics and machine learning, utilities can use advanced analytics to keep their networks’ health at its peak and, in turn, better meet SLA’s service reliability parameters. This use case enables utilities to analyze historical network data collected from multiple sources to anticipate which network elements and/or specific ports are most likely to fail during a given time period. This enables the utility to take proactive measures to avoid service disruptions. Utilities need to specify that their vendors deliver solutions that can provide advanced analytics across multiple domains and vendors. - DS-0 Grooming
Utilities have deployed many lower speed (DS-0) type services over the years and these are not likely to be migrated to packet for some time. For many substations, these services do not even consume the bandwidth of a single T1/E1 so many ungroomed T1/E1 ports waste valuable real estate on substation multiplexers. DS-0 grooming allows these services to be groomed over pseudowires over a packet infrastructure. Utilities should specify that their vendors deliver a solution that grooms both legacy TDM and packet traffic. - MPLS with Dynamic Control Plane
Utilities need to specify the requirement for a low latency, deterministic packet transport network that is demanded by utility applications but takes advantage of Ethernet cost curves and silicon integrated circuits. Examples of this approach include MPLS Tunnels (LSPs) that are built within the metro which then support contained services over Pseudowires (PWs).
