Designlet: HCX Network Extension for Azure VMware Solution

Introduction

HCX Network Extension (NE) provides a Layer 2 VPN (L2VPN) to extend a broadcast domain from a customer site into an Azure VMware Solution private cloud. HCX NE functionality is provided by a dedicated virtual appliance at both sites.

Summary and Considerations

Use Case

VMware HCX Network Extension (NE) is used to provide Layer 2 adjacency between VMs at the customer site and VMs that have been migrated to an Azure VMware Solution private cloud. This provides a stopgap to facilitate communication between VMs in the same VLAN/port group while migrations are occurring. HCX NE is specifically useful for customers who are not able to re-IP VMs during the migration process or migration of the gateway from on-premises to Azure VMware Solution. Additionally, HCX NE can be used in disaster recovery scenarios.

Pre-requisites

  • Working VMware HCX deployment and service mesh
  • VMware NSX-T 3.0 or higher in the Azure VMware Solution private cloud (This is the default deployment for all private clouds after July 23, 2021. For more information see the Azure VMware Solution platform updates.)
  • On-premises vSphere Distributed Switch (vDS) version 5.1.0 or higher required for extending vDS-based networks
  • VMware NSX-V 6.4.8 or higher required for extending NSX-V-based networks
  • VMware NSX-T 2.4 or higher required for extending NSX-T-based networks
  • Existing connectivity to Azure VMware Solution that meets the HCX network underlay minimum requirements

General Considerations/Recommendations

  • A single VMware HCX NE appliance can extend up to 8 networks. HCX manager 4.x supports up to 128 HCX NE appliances.
  • Networks can be extended to a maximum of 3 Azure VMware Solution private clouds.
  • The default gateway for an extended network exists at the customer site. This can lead to sub-optimal routing for cloud based VMs. HCX Mobility Optimized Networking can be used to address this scenario.
  • Do not extend networks used for VMware HCX network profiles, vSphere management networks, or other VMkernel networks (e.g., vMotion/vSAN)
  • VMware HCX NE does not detect or mitigate network loops or IP/MAC conflicts.
  • VMware HCX NE is a tunnel-based technology, which encapsulates traffic between sites. Depending on the MTU of networks in use, packet fragmentation can occur. HCX Traffic Engineering can be used to optimize TCP MSS and reduce fragmentation between VMs connected via NE.
  • Extending vSphere Standard Switch-based networks is not supported
  • VMware NSX networks can be extended. The NSX manager must be registered with HCX to extend NSX networks.

Performance Considerations

An NE appliance is capable of 4-6 Gbps throughput. Additional appliances can be deployed to scale throughput.

Cost Implications

Egress charges may apply to VM traffic on extended networks communicating from Azure VMware Solution to an on-premises environment.

Document Reference

VMware HCX Documentation

VMware HCX Configuration Maximums

VMware HCX Ports and Protocols

Last Updated

August 2021

 

Planning and Implementation

Planning

VMware HCX Network Extension (NE) provides a Layer 2 VPN between a customer site and an Azure VMware Solution private cloud. This service is fully integrated into HCX and provides functionality similar to the NSX L2 VPN. Using an alternative bridging solution, like NSX L2 VPN, is not supported for use with HCX NE. Customers should settle on a single L2 extension technology that meets their migration or disaster recovery needs.

HCX NE appliances are deployed as a pair, with one running at the source site and the other at the destination site. The encrypted tunnel between HCX NE appliances uses UDP ports 500 and 4500. If there are any firewalls in the path between appliances, it should be configured to allow communication between the appliances on these ports.

HCX NE is an optional service, and customers should understand the pros and cons involved with using it. There are alternatives to using HCX NE, like assigning new IPs to VMs as they are migrated or moving a network with all attached VMs to the cloud in a single migration event. HCX NE is a valuable tool when neither of these options is feasible. While the HCX NE appliance is designed for reliability and quick boot, it is not highly available (vSphere High Availability can be used to mitigate this concern.) Additionally, HCX 4.X (starting with HCX 4.0) includes an in-service upgrade option for HCX NE appliances, which significantly reduces the downtime from a software upgrade to a matter of seconds.

Using HCX NE with other HCX services can provide performance benefits and optimizations to traffic flow. HCX Traffic Engineering performs TCP Flow Conditioning, which dynamically adjusts MSS to reduce fragmentation for NE traffic. HCX Mobility Optimized Networking provides optimized traffic flows for VMs that are attached to an extended network and have been migrated to an Azure VMware Solution private cloud.

Diagram, schematic</p>
<p>Description automatically generated

Figure 1: Example HCX Service Mesh with Network Extension

 

Implementation

Eligible networks can be extended via the HCX Manager UI. Follow the steps below to extend a network.

Network Extension

The following steps can be completed using the HCX plugin the vSphere Client or HCX Client

  1. In the HCX Manager UI, navigate to Services > Network Extension. Any existing network extensions are displayed on this screen.
  2. Select Extend Networks
  3. If multiple HCX service meshes exist, select the appropriate service mesh from the dropdown
  4. Select the appropriate network(s) to extend, and click Next
  5. Using the dropdowns, select the NSX-T tier-1 router that the extended network(s) will be attached to, and the HCX NE appliance to use
  6. Provide the gateway IP address and prefix length in CIDR format (e.g., 192.168.10.1/24), and click Submit

HCX will begin the process of extending the network. A status of Extension complete will appear for the network once the network is extended. To verify HCX NE is working, migrate a VM that is connected to an extended network. Once migrated, verify communication is working between the migrated VM and a local VM in the same network. A simple ping should show increased latency to a migrated VM, indicating that the traffic is being transported across the L2VPN tunnel.

Network Extension Details

VMware HCX provides the capability to view information and metrics about extended networks, including local/remote MAC addresses and amount of data transferred.

To view network extension details, follow these steps:

  1. Navigate to Infrastructure > Interconnect
  2. Under the appropriate service mesh, Click View Appliances
  3. Expand the desired network extension appliance, and click Network Extension details
  4. To view metrics and information for a specific network, click Show More Details

Authors and Contributors

  • Emad Younis, Director, Multi-Cloud Center of Excellence, VMware

 

 

 

Filter Tags

Cloud Migration HCX Networking Azure Services Azure VMware Solution Document Technical Guide Advanced Design