Connecting Equinix Direct Connect to VMware Cloud on AWS
This blog will go through the steps to make a direct private connection from Equinix to a VMware Cloud on AWS SDDC. This connection can securely link infrastructure a customer has deployed in different geographical locations.
- Customers Equinix private clouds to the VMware Cloud SDDC.
- Private connections from customers on prem datacenters to VMware Cloud on AWS SDDC through Equinix.
- Edge and ROBO sites to VMware Cloud on AWS SDDC through Equinix.
- Other cloud provider VPC’s, provisioned services and VMware SDDC’s to VMware Cloud on AWS through Equinix.
The customer can link sites to Equinix using private data lines, SD-WAN or VPN tunnels connected to customer owned physical networking infrastructure or use virtual networking devices as a service from Equinix. By using Equinix as the network point of presence or edge route to Hyperscalers like AWS, Google and AVS, a private network connection can quickly be deployed and torn down very easily on demand as this blog will show. Equinix acts as the network hub to link the customer hyperscaler networks on VMware Cloud on AWS, Google Cloud on VMware Engine, AVS as well as between the customer on prem, edge and Equinix located private clouds.
This blog assumes the customer already has a VMware Cloud on AWS SDDC deployed and functional. This blog does not cover the customer side switch and router setup as this can vary depending on the manufacture of the networking equipment. The assumption is the customer already knows how to set up a VLAN with layer 3 routing and can configure the BGP routing protocol on the customer equipment on the Equinix side of the connection. The blog will go over the steps to extend the customers connection using “Equinix Fabric” to an SDDC running in VMware Cloud on AWS.
The following diagram represents a customer with Equinix collocated equipment and on-premises sites connected to Equinix. Equinix fabric is used to create a “Direct Connect” data link to a customer VMware Cloud on AWS SDDC providing a private direct connection for all sites to cloud services in VMware Cloud on AWS. BGP is used between all locations to advertise routes enabling seamless end to end private network environment from on-prem datacenters to edge to Equinix to VMware Cloud on AWS.
The Equinix A and Z side connections provide VLAN translation from your private network VLAN in Equinix to the AWS network VLAN assigned when the “Direct Connect” circuit is provisioned. This provides a seamless layer 2 connection between two different VLAN IDs with Equinix taking care of the translation for the customer.
- New VLAN tagged on customer router on Equinix side fabric connection
- New BGP ASN for use on VMware Cloud on AWS side of connection
- Collect BGP ASN used on customer Equinix side router
- Private /30 IP subnets to use for connection between customer router and VMware Cloud on AWS
Equinix Fabric Connection
The first step to make a direct connection from Equinix to AWS is to login to the Equinix Fabric and Network Edge web portal. Once in the web portal, select “Connections,” “AWS” for the connection, then under “Show” select “Services available to me.” “AWS Direct Connect” click “Create Connection”.
Next click “Create a Connection to AWS Direct Connect”
Now choose “Port” for the type of connection, “Select Location,” where the port will be provisioned, pick the circuit from the available circuit ports in the location and the “Destination” from the available destinations provided by Equinix. Once all information is selected click “Next”
On the connection details page provide a “Virtual Circuit Name,” the “A side VLAN,” the “AWS Account ID” for the link to connect and the “Connection Speed” desired for the circuit. Once all sections are completed click “Next.”
Now review the connection details. If you need to add another email account to get notified about this connection deployment you can add that in the “Notifications.”
If everything is correct after review click “Submit Order.”
Next you will get confirmation that the order has been submitted. You will also receive an email confirmation of the order. Once the line is provisioned you will get an email update to let you know the line is now provisioned and ready.
Once the line is provisioned you will need to login to the Equinix Fabric portal and retrieve the Z side VLAN to use when creating the “Virtual Interface” in AWS.
Equinix translates the VLAN from the A side which is your equipment in Equinix where you can use any VLAN you choose to the Z side of the connection where Equinix designates the VLAN to use with AWS services.
AWS Virtual Interface to VMware Cloud
In the AWS console go to the “Direct Connect” console and click on “Connections”
In “Connections” the new circuit will show up as ordering. This connection needs to be accepted before it will be available to use. Click on the connection ID which will open the properties of this connection.
In the connection properties “Accept” the connection
Before creating a “Virtual Interface” in AWS for the direct connection to VMware Cloud log into the VMware Cloud console and navigate to the SDDC you will be connecting. Under “Inventory,” “Networking & Security,” “Direct Connect” find the “AWS Account ID” and “BGP ASN.” Take note of the “AWS Account ID” to use during the “Virtual Interface” creation. This account ID will link the “Virtual Interface” created in AWS to the VMware Cloud “Direct Connect” interface. Make sure the “BGP ASN” is set to the desired ASN needed for setup on your network. You can only change the ASN when there is no “Virtual Interfaces” attached. Making sure this is correct before attaching the “Virtual interface” will save you from needing to delete and recreate this interface to correct a misconfiguration of the “BGP Local ASN”.
Under “Direct Connect” choose “Virtual Interfaces” and then click “Create virtual interface”
In the “Create virtual Interface” setup. Select “Virtual interface Type” as “Private,” give the “Virtual Interface” a name, choose the “Connection” from the drop down. (This will be the connection previously approved from Equinix”). Select the AWS account that will be used for billing. This will be the “AWS Account ID” previously noted from the VMware Cloud console “Direct Connect”
Next add the “VLAN,” this will be the Equinix VLAN assigned to the Z side of the connection. Set the BGP ASN for the router in Equinix (This will be your A side routers ASN), set the peer router IP (Equinix Router). Now add the AWS router IP address and add the BGP password for your Equinix side router (Note: This is an optional step, but I have not been able to get my routers to peer with VMware Cloud on AWS without authentication. This could be due to my specific setup so this may or may not be required in your environment.) Now click “Create virtual interface”
Once the “Virtual Interface” is created it will go into a state of “pending.”
Go back to the VMware Cloud SDDC console under “Inventory,” “Direct Connect” and there will be a new connection available under “Virtual Interfaces.” Click on “ATTACH” on the new interface and a confirmation window will pop up. In the confirmation window you will be informed that charges may be incurred for the connection and need to agree to any data transfer charges incurred by checking the box, then click the “SAVE” button.
The state of the virtual interface will change to “Attached” and the BGP status should show as “Up.” Learned routes will also populate with routes from your private network.
If BGP does not go to an up-state check that the BGP settings are correct on both sides of the connections. You can also test the layer 2 circuit is connected by pinging the “local IP” listed for the “Virtual Interface” from the remote router in Equinix. You should get a ping reply from the “Virtual Interface” at your Equinix side router even if BGP is in a down state. If you cannot ping, then there is a configuration or a circuit issue to resolve. If you get a reply when you ping across and BGP is not up, then check for issues with the BGP configuration on both sides of the connection.
Other Common Issue Resolutions:
- Connection provisions but not able to communicate over the circuit: Make sure you have provisioned the correct circuit ID that terminates to the correct equipment in your Equinix rack.
- If the “Virtual Interface” does not show up in the VMware Cloud SDDC “Direct Connect”: Make sure you used the correct account when creating the “Virtual Interface”.
- No learned BGP routes: Make sure all the routes intended to be advertised on your Equinix side equipment in the BGP settings are set correctly.
If after troubleshooting the connection, there is still not link, create a support ticket with the appropriate service provider as there may be a service outage causing the issue that needs to be resolved.
Jerry Haskins is a Solutions Architect responsible for collaboration of products on the VMware Partner Solutions Engineering Team in VMware’s Office of the CTO. With 20 years+ of experience in the IT Industry he has spent his career in innovative roles managing enterprise networks and datacenters, working with virtualization technologies, micro services, CI/CD workflows and multi-cloud solutions.