Well-Architected Design: Securing Extended Networks

Introduction

Extending layer-2 networks allows workloads that exist in separate physical locations to communicate over the same subnet, or broadcast domains. It removes the need to re-IP VMs and redesign applications across multiple locations, which can simplify migration operations or data center extensions.

It is important to understand the inbound and outbound traffic flows and how to secure extended networks even if the network extension is only temporarily in place.

Infrastructure Overview

Understanding the traffic flows of an extended network helps identify the right methodology to secure the workloads on that network. VMware Cloud solutions offer several methods to extend networks, such as HCX Network Extension and Layer 2 VPNs. Even though these solutions use different underlying mechanics for extensions, the traffic flows always follow one of two patterns based on where the gateway of the extended network resides.

Network Extension using Source Gateway

In this design, which is also referred to as a traditional extended network, an appliance is deployed to provide a tunneling solution for L2 extension. The extended network is not connected to the gateway at the destination Software-Defined Date Center (SDDC).

Figure 1 – Network extension topology using source gateway.

Since the network is only connected at the source site the source gateway is used for all routing outside of the extended network, even if the workload being communicated with is residing in the same site. This is usually referred to as “tromboning” or “hair-pinning” traffic pattern.

Egress traffic and inbound internet traffic will always be handled at the source site.

Mobility Optimized Networking

Mobility Optimized Networking (MON) is a HCX Enterprise functionality that optimizes traffic flows for extended networks in the destination sites. MON connects the extended network to the destination gateway, but only publishes a /32 route for the MON enabled workloads.

Figure 2 - Work Extension Topology using Mobility Optimized Networking

Since the specific routes of MON enabled workloads are advertised, they can communicate with other networks in the destination site using the destination gateway in the same site. This prevents the traffic hair-pinning and allows for lower latency connectivity between workloads in different networks compared to the previous method mentioned.

To prevent all traffic from using the destination gateway as a default route, MON provides the ability to leverage policy routing to make sure the network extension is still used. Using the diagram in Figure 2 for example, traffic going to network 192.168.10.0/24 should still traverse the network extension tunnel to prevent asymmetrical routing. By default, MON enables policy routing for all RFC 1918 networks.

Egress traffic to non-RFC 1918 networks will leverage the destination gateway. This may cause some issues with stateful firewalls if the inbound and outbound traffic paths are not identical. In that case, all egress traffic can be routed through the source site via policy-based routing.

More information on Mobility Optimized Networking traffic flows can be found in the HCX User Guide.

Operations Overview

Once the traffic paths for the extended networks are understood, the locations and components that can be used to protect the extended workloads can be identified.

Securing Traditional Extended Workloads

When a network is extended in this design, the traffic patterns of extended workloads in the destination SDDC will mostly be identical to workloads residing in the source SDDC. Since egress traffic is forwarded to the source SDDC and the source gateway, any security measures taken in the source SDDC’s traffic path will still be in effect.

For extended workloads in the destination SDDC, the Distributed Firewall (DFW) can also be used to increase the security measures implemented in the source SDDC. In addition to the ability to implement micro-segmentation and traffic within the extended network, unwanted outbound traffic can be blocked closer to the source. This can reduce load on the security appliances in the source SDDC and reduce traffic between sites which may lower egress cost.

Figure 3 – Firewalls Securing Traditional Extended Network

Securing MON Enabled Workloads

The traffic patterns of a workloads connected to a Mobility Optimized extended network vary, which means it can cross different security and networking appliances, both in the source and destination SDDCs. The extended workloads can be routed via the source gateway, the destination gateway, or a combination of both depending on the policy routing configuration, in addition to being routed within the destination SDDC.

As mentioned earlier, outbound traffic to destinations configured in policy routing will follow the same traffic pattern as a traditional extended network. For these traffic flows the same methods that were described in the previous section should be implemented.

When MON is enabled, east-west traffic to and from networks connected to the destination gateway will be routed using the local destination gateway. Since it’s routed locally the destination gateway firewall will not inspect that traffic, which may pose a security vulnerability. The Distributed Firewall should be used to make sure the traffic between workloads within the SDDC, both within the same network and across different networks. Methods to implement the distributed firewall successfully can be found in this Well-Architected Design.

The destination gateway will be used to route outbound traffic if there is a need for local egress of extended workloads. The destination network should not be included in the HCX policy routing configuration to enable this functionality. For these traffic flows the gateway firewall at destination site will be use. The distributed firewall can also be used for more granular rules impacting specific workloads.

Inbound traffic to MON enabled workloads can be configured using NAT at the destination traffic. By default, inbound traffic rules show be configured based on the private IP, but this depends on the NAT configuration in the destination SDDC.

Figure 4 – Firewalls Securing Traditional Extended Network

Protecting the Network Extension Solution

The network extension appliance is a key component in the data path of the extended network. A failure of this appliance will prevent communication between extended workloads in the destination SDDC and the source SDDC. Due to this, network extension solutions included in VMware Cloud supports high availability configurations for the extension appliances.

When enabled, a pair of appliances (a primary and secondary) are deployed to ensure there is redundancy and connectivity is retained in the case of a failure. The impact during a failover varies by environment and depends on the specific network extension solution. In addition to the additional resources consumed by the secondary appliance, other design considerations should be taken into consideration, such as appliance placement. Making sure the appliance pairs reside on separate vSphere nodes on both sites guarantees a node failure won’t impact both appliances. Resources or licensing constraints may also prevent the deployment of a highly available configuration. The availability of the extension appliance can also be increased by leveraging the following design practices:

1. Minimizing frequent vMotion operations of the network appliances can also lower the risk of any impact on the established tunnels between the appliances in the source and destination SDDCs. In a DRS enabled cluster this can be achieved by configuring a soft host-affinity or a partially automated policy for the appliances.
2. Placing the extension appliances on vSphere High Availability enabled clusters will ensure the appliances are rebooted in case of a physical host failure. Based on the criticality of the extended workloads the HA Restart Policy can be increased or decreased to match the extended workload impact.
3. Enabling VM monitoring for the appliances can also instruct vSphere HA to restart the appliances in case of an internal failure of the appliance.