Connecting Data Centers Under the Sea

Brian Lavallée is Director of Portfolio Solutions Marketing for Ciena's packet and submarine networking solutions.

Thousands of years ago, our early descendants settled near bodies of water and erected small settlements that would flourish over centuries into the large metropolises of today. They settled near bodies of water because nearby land was fertile so crops could be grown, farm animals could be fed and raised, and the body of water itself, at least the fresh water variety, provided a steady supply of drinking water. These bodies of water would eventually be used for transportation between settlements leading to commerce and communications. Major waterways soon became primary trade routes with the shortest path typically taken for the quickest, and often safest, journey. As cities grew in size, their ports became hubs of commerce for the exchange of goods and services. So, what does this all have to do with data centers? Bear with me, as I’m getting to it.

Today’s commercial shipping routes commonly take the shortest path between ports in major cities, which minimizes fuel, travel time, and associated costs. Likewise, submarine cables follow similar paths connecting the same major cities to minimize the amount of submarine cable hardware for minimized cost and travel time (latency). Perhaps this is why the vast majority of submarine cable faults are attributed to marine vessels.

Directly connecting cities with submarine cables made sense, as most early traffic was voice and cities are where most people live. However, much has changed in the past decade with data now dwarfing voice on submarine cables. Where’s this submarine data traffic coming from and where’s it going? It’s between the massive data centers owned and operated by large Internet Content Providers (ICPs) for such content-related applications as backup, restore, caching, load balancing, and synchronization. In short, data centers talk to each other, a lot.

The ICP Effect on Submarine Cables

ICPs have an enormous impact on the amount of traffic carried by submarine cables. While bandwidth used for the public Internet has accounted for the majority of international bandwidth usage for well over a decade, a momentous shift is underway. Private networks, led by major ICPs, are adding capacity at a quicker pace than traditional Internet backbone operators.

According to respected analyst firm TeleGeography, private network bandwidth along the trans-Atlantic corridor connecting North America to Europe exceeded Internet bandwidth for the first time in 2014. They forecast that by 2019, private networks will account for the majority of overall international bandwidth usage. So, landing new submarine cables in or nearby large cities does indeed make sense, as long as the data centers are actually located within these cities.

Data centers were initially located close to end-users in densely populated cities; however, this is decreasingly the case, as data centers are being built far away from cities to take advantage of more favorable regulatory environments, lower energy and real estate costs, or other cost-saving reasons, such as tax breaks. Large data centers have even been built in remote Nordic locations in Sweden and Finland to take advantage of the cold air to cool data centers and further save on energy costs. Iceland by virtue of its geothermal power, which is abundant, cost-effective, and environmentally friendly, suddenly becomes a prime location for data centers. Regardless of where these data centers are being located, the network ultimately dictates the user experience.

Users, man or machine, must have access to content stored in remote data centers, which necessitates high-speed network connectivity, which drives new submarine cable builds. Large data centers situated far from city centers challenge the traditional landing of submarine cables in or very close to cities.

Inland terrestrial networks can then connect users to the content within these data centers. Although new submarine cable landing locations are (very) difficult to get approved, I believe ICPs have the leverage to get it done, if so inclined. Time will tell.

For submarine cables targeted primarily at DCI applications, it makes sense to consider moving Submarine Line Terminating Equipment (SLTE) out of traditional Cable Landing Stations (CLS) and directly into the data center. This is a viable option when using the latest generation of SLTE that is capable of crossing oceans with sufficient remaining margin to go hundreds of kilometers further inland on both ends of the submarine cable. Ciena calls this POP-to-POP network architecture GeoMesh that initially united terrestrial and submarine networks. Some equipment will remain in the CLS, such as Power Feed Equipment (PFE), but moving SLTE inland makes sense.

ICPs now play a prominent role in submarine network development and will increasingly participate as anchor investors and/or consortium members in new submarine cable builds. They may also ultimately decide to build their own private submarine cables as well. Given the ICP’s affinity for open networks, as evidenced by the Open Compute Project and Telecom Infra Project, its no surprise they are vocal advocates of Open Cables, which aim to separate wet plants from SLTE. This enables choice where they can choose from a broad selection of vendors the best-in-breed wet plant, SLTE, and Network Management Systems (NMS).

Open submarine networks are achieved through the adoption of open Application Programming Interfaces that virtualize submarine networks assets. This is similar to terrestrial networks being virtualized with the advent of SDN. Given the POP-to-POP nature of DCI connectivity, all parts of the end-to-end network must be seamlessly interconnected and virtualized, and is the meaning behind the “Land. Sea. Cloud. Networks Unite.” tagline associated with Ciena’s GeoMesh, which has been deployed worldwide and continues to evolve.

I’ve talked about the networks that interconnect data centers being submerged, so why not submerge the data centers themselves? Microsoft is already studying this question with their innovative Project Natick. Microsoft says 50% of us live near the coast, so why doesn’t our data? This is indeed a thought-provoking question. ICPs have already had a profound impact on the storage and compute industries and are having a similar effect on terrestrial and submarine networking industries. As DCI traffic continues to dominate, traditional mindsets of the submarine network industry will surely be challenged by ICP heavyweights leading to further innovations.

This is a good thing.

For more information about GeoMesh, download the Ciena white paper.