I will be starting a series of posts on NSX-T Edge Node design topologies.
In this blog post I will start with a Edge Node design topology hosting a Tier-0 Gateway with Static Routing with a HA VIP address configured.
With the Edge Node Virtual Appliances it is important to know on which vSphere Hosts the Edge Nodes are going to run, how many physical NICs are available and if the Edge Node is running on top of a VSS/VDS or N-VDS and how teaming is configured.
The following topology is showing two Edge Nodes running on a vSphere Cluster with hosts configured with one VDS with two uplinks sharing Management, Overlay and Uplink connectivity.
On the Top of Rack (ToR) switches VLAN 100 is configured as Geneve transport network to carry Overlay traffic and VLAN 101 is configured as Transit Network for North-South traffic to/from the physical network towards/from the Tier-0 Gateway.
The Edge Node VM must have Management, Overlay, and Uplink Router (North/South) connectivity. The management IP can use the same Port Group as ESXi hosts or any general management Port Group. In this example there is a Port Group created for Overlay (Overlay-PG) and one for External traffic (External-A-PG). Additionally it is recommended to create a ‘null’ portgroup and connecting this to the last Edge Node vNIC to suppress alerts in NSX-T if the Edge Node has no connection on that interface.
I recommend to configure specific Failover Order on the Port Groups for achieving deterministic traffic and symmetric bandwidth for both Overlay and North-South traffic. In this topology this would make sure that Overlay traffic from fp-eth0 on the Edge Node, using the Overlay-PG, will go through Uplink-1 (vmnic0) and North-South traffic from fp-eth1, using the External-A-PG, will go through Uplink-2 (vmnic1).
In NSX-T two Transport Zones (Overlay & VLAN Transport Zone) will share the same N-VDS (N-VDS-01) and a additional Transport Zone with N-VDS (N-VDS-External-A) is required to connect the fp-eth1 interface to.
I recommend tagging VLANs as much as possible in NSX-T to have the best networking visibility in NSX-T. Doing so requires you to create the Port Groups on the VDS as Trunks allowing all or a range of VLAN IDs. In NSX-T you need to configure the VLAN (VLAN 100) for Overlay traffic in the Uplink Profile for the Edge Node. Additionally you need to configure a Segment with the VLAN ID associated with the North-South transit network, in the External-A Transport Zone, on which the Uplinks of your Tier-0 Gateway will connect (VLAN 101).
When configuring HA VIP address this means that two Edge Nodes in an Edge Cluster are in Active/Standby configuration and one of the two Edge Nodes is actually forwarding traffic, as shown below. When the Active Edge Node fails the Standby Edge Node will detect that by means of HA Heartbeat mechanism and take over the role as Active Edge Node.
Links to other NSX-T Edge Design Posts