This is the next post on a series of posts on NSX-T Edge Node design topologies.
In this blog post I will describe an Edge Node design topology hosting a Tier-0 Gateway with Static Routing with a HA VIP address configured.
With the Edge Node Virtual Appliances it is important to know on which vSphere Hosts the Edge Nodes are going to run, how many physical NICs are available and if the Edge Node is running on top of a VSS/VDS or N-VDS and how teaming is configured. In this design topology the Edge Nodes are running on hosts with two VDS with four Physical NICs.
The following topology is showing two Edge Nodes running on a vSphere Cluster with hosts configured with two VDS with four uplinks. These topologies are very common on environments where customers want to separate Management, vMotion and in some cases Storage traffic, from VM traffic.
On the Top of Rack (ToR) switches VLAN 100 is configured as Geneve transport network to carry Overlay traffic and VLAN 101 is configured as Transit Network for North-South traffic to/from the physical network towards/from the Tier-0 Gateway.
The Edge Node VM must have Management, Overlay, and Uplink Router (North/South) connectivity. The management IP can use the same Port Group as ESXi hosts or any general management Port Group. Note that in this topology the Management Port Group is in VDS-01, which is dedicated for Management connectivity. In this example there is a Port Group created for Overlay (Overlay-PG) and one for External traffic (External-A-PG) on VDS-02. Additionally it is recommended to create a ‘null’ portgroup and connecting this to the last Edge Node vNIC to suppress alerts in NSX-T if the Edge Node has no connection on that interface.
I recommend to configure specific Failover Order on the Port Groups for achieving deterministic traffic and symmetric bandwidth for both Overlay and North-South traffic. In this topology on VDS-02 this would make sure that Overlay traffic from fp-eth0 on the Edge Node, using the Overlay-PG, will go through Uplink-3 (vmnic2) and North-South traffic from fp-eth1, using the External-A-PG, will go through Uplink-4 (vmnic3).
In NSX-T two Transport Zones (Overlay & VLAN Transport Zone) will share the same N-VDS (N-VDS-01) and a additional Transport Zone with N-VDS (N-VDS-External-A) is required to connect the fp-eth1 interface to.
I recommend tagging VLANs as much as possible in NSX-T to have the best networking visibility in NSX-T. Doing so requires you to create the Port Groups on the VDS as Trunks allowing all or a range of VLAN IDs. In NSX-T you need to configure the VLAN (VLAN 100) for Overlay traffic in the Uplink Profile for the Edge Node. Additionally you need to configure a Segment with the VLAN ID associated with the North-South transit network, in the External-A Transport Zone, on which the Uplinks of your Tier-0 Gateway will connect (VLAN 101).
When configuring HA VIP address this means that two Edge Nodes in an Edge Cluster are in Active/Standby configuration and one of the two Edge Nodes is actually forwarding traffic, as shown below. When the Active Edge Node fails the Standby Edge Node will detect that by means of HA Heartbeat mechanism and take over the role as Active Edge Node.
Links to other NSX-T Edge Design Posts