Understanding VPN to Customer Created NSX T1s in VMC on AWS

December 02, 2022

Introduction

VMC on AWS added the capability to configure customer created NSX T1s. These new T1s enable multiple use cases including but not limited to, network centric multi-tenancy, application tenancy and disaster recovery testing.  A common requirement is to provide VPN access to the tenant’s T1 and networks behind it.  This specific use case will be reviewed in this document. Customer created T1s can be configured in the VMC Networking & Security tab -> Tier-1 Gateways with limited capabilities. Full capabilities are available in the NSX Manager UI -> Networking -> Tier-1 Gateways or via API.

General Considerations for VPN to T1 Gateways

  • SDDC must be version 1.18 or higher
  • Route-based VPNs do not support a dynamic routing protocol, but do support static routes
  • Route-based VPNs do not support VPN redundancy
  • The default route (0.0.0.0/0) cannot be configured to use the Virtual Tunnel Interface (VTI) on a T1 Gateway in Routed or NATted mode
  • The VPN Local Endpoints (LEPs) require a route aggregation if the VPN is traversing an AWS Direct Connect or VMware Transit Connect
  • The VPN LEPs do not require a route aggregation if the VPN uses the Internet as its transport via an AWS Public IP
  • Using an IP address from the SDDC Management CIDR for a VPN LEP is not supported
  • Policy VPN with NAT requires SDDC 1.20 or higher

Documentation Reference

Multiple CGWs in VMC on AWS

Understanding Route Aggregation in VMC on AWS

ConfigMax

Last Updated

January 2023

Considerations

Configuring VPNs on customer created T1s enables additional use cases like application tenancy or duplicate/overlapping IP addresses in the SDDC for VMware Cloud on AWS. Customer created T1s support three types of VPN:

  • Layer 2
  • Policy-Based
  • Route-Based

This article will discuss policy and route-based VPNs. From a NSX architecture perspective, VPNs will terminate on an IP address referred to as the Local Endpoint (LEP). The LEP is associated with the T1 it is created on and must be a unique IP address in the network. Internet Key Exchange (IKE) and Encapsulated Security Protocol (ESP) will use the LEP as the source and destination addresses for IPsec session. The diagram below illustrates the LEP, Virtual Tunnel Interface (VTI) and remote ID usage.

Anatomy of a VPN

NOTE – It is important to differentiate between policy-based and route-based VPNs. A route-based VPN creates a single SA pair, also referred to as a tunnel, for all traffic, and then uses an IP interface on each end as a route destination. The routing can be dynamic (BGP, OSPF) or static. In the T1 VPN, only static routes are supported. Policy based VPNs do not use routing at all, but instead rely on policy matching applied to an interface to determine the traffic sent over the VPN tunnel. As such routing principles like shortest-prefix-wins do not apply. In general Policy VPNs require more configuration as both sides must be updated to reflect changes in the network topology.

Performance of T1 VPNs is a component of overall VMC on AWS networking performance. VMC on AWS uses a single edge cluster for all VPNs including T0 and T1. As a cumulative total, all VPNs performance is shared. In environments where the requirement for higher capacity VPNs is needed (greater than 2Gbps) large Edge appliances are recommended. A good reference for network performance in VMC on AWS can be found here in the VMC on AWS Management Cluster Planning designlet.

Configuration Steps

Configuring a VPN is a multi-step process with both policy and route-based VPNs sharing many of the same steps for configuration. This document will review three examples of VPNs.

  1. Route-based VPN over the Internet
  2. Route-based VPN over an AWS Direct Connect or VMware Transit Connect
  3. Policy based VPN over an AWS Direct Connect or VMware Transit Connect

Common VPN configuration steps must be completed in order as the step build upon themselves.  The flowchart below outlines the process.

T1 VPN Flowchart

We will follow this process on an Internet based VPN in the example below. The topology used is a remote NSX VPN endpoint using the Internet to establish a route-based VPN to the Tenant-A Tier-1 router and is illustrated below.

Step 1 – Request Public IP address from AWS

Request Public IP

Step 2 – NAT configuration on the NSX T0

In this configuration the LEP will be configured with the public IP address allocated from AWS. This is done to minimize confusion with address translation, but another address could be used for the LEP and NATted accordingly. It is recommended to use the public IP address for the LEP in a NoNAT configuration. NoNAT refers to a NAT configuration where the IP address is consistent end to end and not translated.

T1 VPN NAT Configuration

 

Step 3 – Create IPsec service

T1VPN IPsec Service creation

Step 4 – Create the Local Endpoint (LEP)

T1 VPN Local Endpoint Creation

Step 5 – Configure IPsec session

T1 VPN IPsec Session creation

Step 6 – Optional - Configuration of IPsec profiles

VMC on AWS has default IKE, IPsec and Dead Peer Detection (DPD) Profiles defined as shown below. If the use case requires different parameters, custom profiles can be configured and used.

Default IKE profile

Default IKE Profile

Default IPsec profile

Default IPsec profile

Default Dead Peer Detection (DPD) profile

Default DPD Profile

Step 7 – Create Default CGW Firewall rules

The SDDC1-Tenant-A-LEP-NAT and SDDC2-Tenant-A-LEP-NAT groups contain their respective LEP IP addresses. These are not auto-created and the three protocols shown are the minimum required for the VPN to come up and pass traffic.

T1 VPN Default CGW FW Rules

Step 8 – Set static route(s) on T1

The next hop is the IP address of the remote side of the virtual tunnel interface (VTI). In the use case for policy VPN, static routes are not needed.

T1 VPN Set Static Routes

Using the same flowchart for a route based VPN over an AWS Direct Connect or VMware Transit Connect . In use cases where the VPN uses AWS Direct Connect or VMware Transit Connect, be sure to configure route aggregation for the VPN LEP on the INTRANET connectivity endpoint.

Step 9 – Create Aggregation Prefix List and apply

T1 VPN Route Aggregation

Summary

The addition of VPNs directly to customer managed T1s opens new connectivity and tenancy models in VMware Cloud on AWS SDDCs.

Filter Tags

Cloud Migration DC Extension DRaaS General Networking NSX VPN VMware Cloud VMware Cloud on AWS Blog Designlet Technical Guide Technical Overview Designlets Networking Security Intermediate Design Deploy Manage