Well-Architected Design: Gateway Firewall Considerations
Introduction
A VMware Cloud Software-Defined Data Center (SDDC) includes vCenter Server, NSX software-defined networking, and vSAN software-defined storage. These products are delivered as a cloud service by VMware or one of its hyperscaler partners to accelerate cloud adoption and simplify the cloud operating model. VMware Cloud SDDC simplifies the consumption of advanced networking capabilities using NSX, and customers must understand the networking and security model of VMware Cloud SDDC. This design will help to adopt a modern security architecture with consistent protection across on-premises and cloud environments.
Scope of the Document
Within a VMware Cloud SDDC, NSX provides two layers of intrinsic network security: the NSX Gateway Firewall and the NSX Distributed Firewall (DFW). These two firewalls extend the capabilities to provide defense-in-depth protection across the entire VMware Cloud SDDC infrastructure. This document will focus on the NSX Gateway Firewall design considerations in the VMware Cloud SDDC.
Infrastructure Overview
NSX includes two types of firewalls within the VMware Cloud SDDC: gateway firewalls and distributed firewalls. Gateway firewalls provide north-south protection for the SDDC's zones, perimeters, or boundaries, whereas distributed firewalls provide east-west protection for workloads at the vNIC level.
Figure 1 - SDDC Firewall Types
NSX Firewalling Approach
VMware NSX Gateway Firewall is a software-defined, layer-4 firewall that enables you to achieve consistent network security coverage and unified management for all your workloads, whether running a VMware-based cloud infrastructure solution on a private or public cloud environment or in containers. When deployed in conjunction with the NSX Distributed Firewall, the Gateway Firewall extends its capabilities to deliver consistent protection across the entirety of the infrastructure.
NSX Gateway Firewall
NSX Gateway Firewall is enforced on Tier-1 or Tier-0 routers and provides essential perimeter L3/L4 stateful firewall protection for north-south traffic (traffic leaving or entering the VMware Cloud SDDC). It's important to mention that the gateway firewall enforces traffic protection on the router uplink, making it incapable of protecting east-west traffic between workloads.
NSX Distributed Firewall
VMware Cloud SDDC adds another layer of firewalling to secure east-west traffic. The traffic could be within a specific segment, between two segments connected to the same Tier-1 Gateway, or even within the entire SDDC. Distributed Firewall rules apply at the VM (vNIC) level built into the hypervisor and delivered at each workload, enabling it to enforce access controls and inspect every flow for threats without traffic hair-pinning to a gateway firewall. It simplifies operationalizing east-west traffic security at scale. For more information on the uses cases, see the NSX DFW Use Cases Design.
Figure 2 - NSX Firewall
Operations Overview
With VMware Cloud SDDC, customers are responsible for configuring the gateway firewall and implementing security policies. The NSX recommendation is to design a least privileged access model with workloads having access to only what is required. Examples of this model include using trusted sources, destination IP addresses, and destination ports through NSX grouping constructs like tags, groups, and the "Applied-To" field.
Consider the following operational best practices when implementing your firewall design:
- Implement Identity Governance and Administration (IGA) for NSX service role access to obtain data for audit trail and set policies to control and manage API tokens access in real-time.
- Implement multi-factor authentication.
- Monitor NSX audit logs for network changes.
- Enable NSX firewall logs for visibility and troubleshooting.
- Archive NSX logs for security compliance.
Design Considerations for Management Infrastructure Firewall Policies
Management of your VMware Cloud SDDC is performed through interfaces provided by vCenter and NSX Manager. You can access the vCenter and NSX Manager in your VMware Cloud on Public Cloud SDDC in several ways:
- By using a public IP address reachable by any browser that can connect to the Internet
- By using a private IP address through a VPN or a dedicated high bandwidth, low latency connection
- By using a bastion / jump server located in the public cloud native instance.
Figure 3 depicts the SDDC Management Infrastructure security in VMware Cloud on AWS provides Management Gateway (MGW), a Tier-1 Gateway that handles routing and firewalling for vCenter Server and other management appliances running in the SDDC. The Management Gateway Firewall runs on the MGW to protect management VMs for north-south security and from compute networks within the SDDC. Since keeping your SDDC management infrastructure safe and secure is critical, the management gateway, by default, blocks traffic to all management network destinations from all sources until the customer creates a Management Gateway Firewall rule allowing access from a trusted source.
When configuring access to the SDDC management infrastructure, you must create management gateway firewall rules that allow only the necessary access to the SDDC management network.
Figure 3 - SDDC Management Infrastructure Security in VMware Cloud on AWS
|
Design Consideration |
Design Justification |
Design Implication |
|
Plan to configure access to SDDC management infrastructure such as NSX Manager and vCenter with appropriate gateway firewall security.
|
The gateway firewall blocks traffic from all sources to all management network destinations by default. Ensure gateway firewall rules only allow traffic from IP addresses you own or a trusted source. Always limit the source IP ranges, internal and external, to the smallest possible set. Prevent untrusted sources from accessing the management network.
|
Temporary access to Management Infrastructure may be required over the Internet when setting up a new SDDC until you can establish permanent connectivity. You should consider creating granular temporary access rules in those scenarios. For example, an enterprise that accesses the Internet from an address in the CIDR block 93.184.216.34/30 should create a management gateway firewall rule that allows only traffic with a Sources CIDR of 93.184.216.34/30 to access management destinations. |
|
Plan to configure access to allow internal workload VMs to access management VMs |
In the default configuration, gateway firewall rules prevent VMs on the compute network from accessing VMs on the management network. Ensure gateway firewall rules only allow traffic from IP addresses of the workload VMs to limit the source IP or IP ranges to the smallest possible set. |
SDDC management infrastructure components such as NSX Manager and vCenter should have the highest security level. Maintaining the safety and security of your SDDC management infrastructure is critical. Ensure only authorized workloads VM are allowed and are granted the least privilege to the management appliances. |
Management Infrastructure Firewall Use Case
Some standard firewall rule configurations include opening access to the vSphere Client from the Internet, allowing access to vCenter Server through the management VPN tunnel, and allowing remote console access.
The following table shows examples of commonly used firewall rules with service, source, and destination settings.
|
Use Cases |
Service |
Source |
Destination |
|
Provide access to vCenter Server from the Internet. Use for general vSphere Client access as well as for monitoring vCenter Server. |
HTTPS |
IP address or CIDR block from on-premises data center. Important: Although you can select Any as the source address in a firewall rule, you cannot use any of the wildcards 0.0.0.0/0 as the source address when the destination is vCenter. Doing so can open access to your vCenter Server and compromise your SDDC. Starting SDDC version M22 in VMware Cloud on AWS, ANY or the default address (i.e., 0.0.0.0/0) will not be accepted as a source. |
vCenter |
|
Provide access to vCenter Server over VPN tunnel. Required for Management Gateway VPN, Hybrid Linked Mode, and Content Library. |
HTTPS |
IP address or CIDR block from on-premises data center. |
vCenter |
|
Provide access from the cloud vCenter Server to on-premises services such as Active Directory, Platform Services Controller, and Content Library. |
Any |
vCenter |
IP address or CIDR block from on-premises data center |
|
Provisioning operations involving network file copy traffic, such as cold migration, cloning from on-premises VMs, snapshot migration, replication, etc. |
Provisioning |
IP address or CIDR block, either public or from an on-premises data center connected by a private tunnel |
ESXi Management |
|
VMware Remote Console (VMRC) access
|
Remote Console |
IP address or CIDR block, either public or from an on-premises data center connected by a private tunnel |
ESXi Management |
|
vMotion traffic over VPN |
Any |
ESXi Management |
IP address or CIDR block from on-premises data center |
Design Considerations for Tier-1 Gateway Firewall Policies
The Tier-1 gateway is preconfigured in your SDDC. In a new SDDC, the default firewall rule blocks traffic to all uplinks in the NSX Gateway Firewall. New SDDCs are created without a default network segment, so customers must create at least one compute segment for their workload VMs and add Gateway Firewall rules to allow traffic as needed.
The Tier-1 Gateway Firewall provides firewalling services at the border or perimeter of workload network segments to secure north-south traffic leaving or entering the gateway firewall. It also provides VPN, NAT, and DHCP services.
Figure 4 - Tier-1 Gateway Firewall
|
Design Consideration |
Design Justification |
Design Implication |
|
Plan to configure gateway firewall to control traffic to and from the external uplinks to compute workload segments behind the same Tier-1 Gateway. |
The gateway firewall specifications and functionalities are appropriate for firewalling at the perimeter. Gateway firewalling at the perimeter allows for a coarse-grain policy definition and provides security policies that are valid for all the workloads and segments connected to the Tier-1 Gateway, which can significantly reduce the security policy counts. |
The gateway firewall enforces traffic control on uplinks interfaces only. Gateway Firewall does not control traffic between segments or workloads connected behind the same Tier-1 Gateway. Workloads virtual machines are attached to the same Gateway network, and all traffic will be allowed between these segments by default. You must use NSX Distributed Firewall to apply rules to restrict the traffic between the network segments, the VMs, and the required network ports. Distributed Firewall (DFW) provides a fine-grain policy for granular or Micro-segmentation definition that enables a zero-trust approach. |
On-Premises data center to VMware Cloud Use Case
VMware delivers a consistent foundation for your workloads on-premises and through VMware Cloud offerings. By deploying VMware Cloud everywhere, you can manage infrastructure and applications in the cloud with the same tools, teams, skills, policies, and standards you use today in the data center. You get a single operating model based on the world's most proven and widely deployed cloud infrastructure for all your applications. And VMware is committed to cloud solutions that ensure you can support modern applications now and in the future. VMware Cloud solutions create new possibilities for IT organizations, opening the door to cost-effective scalability, flexibility, and modernization.
VMware Cloud deployments integrate public cloud and on-premises computing through hybrid connectivity options such as dedicated connection, VPN, and HCX. Once connected, you can apply the same security posture for your workload VMs on-premises or in a VMware Cloud.
The following outlines an example deployment topology, including on-premises and a VMware Cloud on AWS deployment.
Use Case Details
The following is an example of deployment topology, including on-premises and a VMware Cloud on AWS deployment.
- On-premises network segments require access to a VMware Cloud segment.
- On-premises – TEST (172.16.2.0/24) DEV (172.16.1.0/24)
- VMware Cloud – SDDC (192.168.10.0/24)
- Common shared services require access from on-premises to VMware Cloud.
- DCHP, DNS, NTP, etc.
- To allow for these use cases, we must create firewall rules on-premises and in the VMware Cloud SDDC, as outlined below.
Figure 5 - Hybrid Cloud Connectivity Considerations
|
Use Case |
On-premises Recommendations |
VMware Cloud Recommendations |
|
Using gateway firewall when on-premises workloads are connected to VMware Cloud. |
Check the gateway firewall rules at on-premises for 172.16.1.0/24 DEV, 172.16.2.0/24 TEST access to 192.168.10.0/24 |
Create a gateway firewall rule for all traffic from 172.16.1.0/24 DEV to 172.16.2.0/24 TEST access to 192.168.10.0/24.
|
|
Using gateway firewalls with VMware Cloud when DNS, DHCP, NTP, and other common shared services are running on-premises. |
DNS, DHCP, NTP, etc, are running on-premises. Create a firewall rule to allow traffic from VMware Cloud segments/workloads to these services. |
Create a gateway firewall rule for all traffic between Workload VMs and DNS, DHCP, and NTP Servers.
|
Design Considerations for Multiple Tier-1 Gateways Firewall Policies
The NSX Gateway Firewall includes platform capabilities such as high availability and native multi-tenancy support to operationalize multi-tenant deployments quickly. Additional Tier-1 gateways allow an SDDC network administrator to dedicate workload network capacity to specific projects, tenants, or other administrative units within a VMware Cloud SDDC.
Figure 6 shows the topology with the multiple Tier-1 gateways. Each Tier-1 Gateway has its own Gateway Firewall that protects the compute network segments and connects the SDDC Tier-0 Gateway.
Figure 6 - Multiple Tier-1 Gateways
| Design Considerations | Design Justification | Design Implication |
| Plan to configure the gateway firewall to control traffic to and from the compute workload segments behind the additional Tier-1 Gateways. | The additional Tier-1 Gateway has its own firewall or security rule table. Individual gateway firewall enforces security policies at the gateway level for all traffic entering or exiting the gateway. Multiple gateway firewalls make it easy and ideal for creating security zones according to tenant or network boundaries and managing firewall rules per zone. | For firewalling between zones, the gateway firewall only enforces traffic control on uplinks interfaces of those additional Tier-1 Gateways.
Customers create additional Tier-1 Gateways and manage the life cycle for those Tier-1 Gateways. |
Tier-1 Gateway Firewalls as Tenant or Zone Firewalls Use Case
The Tier-1 Gateway applies the policy at the gateway level for firewalling between tenants. Figure 7 shows the example of three individual Tier-1 Gateways providing tenants or zones. In this figure, Tenant A, Tenant B, and Tenant C have overlapping IP addresses, allowing to design zones for different use cases. In this example, the Tier-0 gateway is also an inter-tenant connector.
Figure 7 - Tier-1 Gateway as Tenant or Zone Firewalls Topologies
| Use Cases | VMware Cloud Recommendations |
| Tenant-A has the following objectives:
| The additional Tier-1 Gateway using the "Routed" type provides routing and firewalling services scope for its tenant segment traffic. Distributed Firewall is required to filter traffic between two segments connected to the same Tenant-A Tier-1 Gateway. Critical workload may require a more restrictive policy. The Distributed Firewall can extend its capabilities with advanced security, including NSX Distributed IDS/IPS. |
| Tenant-B has similar objectives as Tenant-A but with additional requirements to use overlapping IP address space across Tenants or Zones. For example, the workload portability use case simplifies application migration across on-premises and cloud environments. | The additional Tier-1 Gateway using the "NATted" type provides tenant segment traffic to traverse using NAT rules created on Tenant-B Gateway, as shown in Figure 7. Local segments can communicate between them behind the NATted gateway. The routing table will not include those segments in Tier-0 that enable overlapping CIDRs in the SDDC. The recommendations for the NATted Gateway are similar to those for the Routed Gateway. |
| Tenant-C requires isolation from the rest of the SDDC network. For example, isolated zones use cases such as Disaster Recovery (DR) testing or "Sandbox" environments. | The additional Tier-1 Gateway using the "Isolated" type will not allow tenant segment traffic to traverse outside. Local segments can communicate between them behind the isolated gateway. The routing table will not include those segments in Tier-0. |
Troubleshooting Considerations
| Troubleshooting Considerations | Recommendations |
| Ensure NSX Gateway Firewall logging is enabled for auditing, monitoring, and troubleshooting purposes. | Gateway Firewall Logs are available in VMware Aria Operations for Logs integration with VMware Cloud SDDC. VMware Aria Operations for Logs provides unified real-time visibility into the VMware Cloud SDDC environment. NSX network packet logs for both gateway firewall and distributed firewall allow admins/application owners to troubleshoot/application monitoring along with security monitoring and log packets for specific firewall rules to accelerate troubleshooting and maintain security monitoring. |
| Gateway firewall logging is disabled by default and can only be enabled per rule. | Enable logging rule by rule based on specific log requirements. Firewall logs can be enabled/disabled via the NSX Manager console, and admins will automatically start seeing firewall logs, allowing them to audit, monitor and troubleshoot the VMware Cloud environment. |
| Ensure NSX Audit Logs are collected for auditing, monitoring, and troubleshooting purposes. | By default, Audit logs are collected in VMware Aria Operations for Logs for all the SDDC deployed in the VMware Cloud organization. |
| Troubleshooting any network issues due to any network change. | Monitoring Logical Network (Segments) changes by automatically forwarding the logs to Aria Operations for Logs when a logical network was created, changed, or deleted. It also allows you to audit when the networks are created in the VMware Cloud SDDC. Monitor Firewall Rule changes when a firewall rule is created, changed, or deleted, which helps security monitor and keep the VMware Cloud SDDC secured. Monitor NAT rules change when a NAT rule is created, changed, or deleted, which helps in application monitoring to analyze traffic sources and destinations for the application traffic. |
| Visibility and Self-serve Troubleshooting | Utilize the NSX Traceflow tool to visualize the path of a packet from any source to the destination for connectivity issues in the SDDC. It helps identify the hop-by-hop path, including gateway firewall and distributed firewall rules that affect the flow of packets in the SDDC. Traceflow is a built-in utility in the NSX manager via UI/API on VMware Cloud SDDC. |
| Traffic that is intended to pass through is being blocked. | Perform the Traceflow with the source to the destination information. Traceflow displays gateway firewall and distributed firewall rules as intermediate hops for the traffic passing or dropping with the reason for failure. Traceflow (vmware.com) |
| NSX Intelligence/Aria Operations for Log/Aria Operations for Networks | Discovers the applicable policies, groups, and services and detects suspicious activity. Check NSX Intelligence/Aria Operations documentation for additional details. |