What is the Underlay Network?
The underlay refers to the physical cabling medium that actually connects the switches and routers in the network. This is essentially referring to Layer 1 in the OSI model - The Physical layer.
In Cisco SDA the controller (Catalyst Centre/DNAC) there are two ways to build an underlay.
- LAN Automation - This feature in Catalyst Center allows for provisioning of the whole Underlay Fabric. It pushes out the IS-IS config to the SDA nodes participating in the Fabric. IS-IS is typically the protocol that is pushed out to underlay devices. The goal of IS-IS here is to provide Loopback-to-Loopback connectivity between the fabric node switches.
- Manually Configuration - It is possible to manually configure the full SDA underlay fabric manually. This can be achieved through CLI or via Automation such as Ansible or Terraform. However, this is time consuming and prone to human error, or bugs when using automation.
Tip
LAN Automation only applies to nodes in the Fabric. Any nodes outside the fabric such as Fusion Firewalls/Routers do not get this underlay configured.
This will be discussed in further detail in another blog regarding the Fabric Node roles.
The Underlay Nodes transports data packets between network devices in the SDA Fabric. Since the Underlay is what transports the packets, any negative impact on the Underlay (such as a failed device) will cause issues in the transmission of data packets. It is recommended to have the Underlay configured with Switch Stacking, High Bandwidth Capability, and High Availability.
Examples of Underlay & Overlays
GRE Tunnels
Generic Routing Encapsulation is one of the most common use case of Underlay & Overlay. This was how I learnt this concept.
Lets consider this network topology.
We have two sites -London & LA- and a DC. CSW1, CSW2, R1 and R2 are in our control alongside the DC_R1 router.
Between London & LA we have a shared medium that is connecting them..the internet. The internet is not in our control.
Typically if we want to run a routing protocol such as OSPF between R1 and R2, we cannot run just pure IP over the internet. Over the internet we typically use a transport mechanism such as TCP or UDP to transport our IP packets. OSPF runs over IP protocol 89 and typically cant be run over the internet, and even if you could, OSPF uses multicast and needs physical connectivity to the other side to form an adjacency. Since its the internet connecting these remote sites we do not have direct one to one physical connection. There are a few solutions to this including free and paid services. GRE is one of the free solutions to allow R1 and R2 to communicate OSPF packets.
How GRE Works
GRE creates a logical Point-to-Point link between R1 and R2. This is a virtual logical link. This runs on top of the physical underlay. As long as there is IP connectivity from R1 and R2 we can now share the routing packets. It doesn't matter if there is a static route on our end to reach R2 or via BGP, as long as there is IP reachability then GRE will work and connect these two nodes.
We direct which traffic goes over the tunnel using static routes. For example:
(Create the tunnel) (WIP on config)
172.16.1.1 is Logical Tunnel Interface on R1
172.16.1.2 is Logical Tunnel Interface on R2
R1(config)#ip route 10.2.0.0 255.255.255.0 172.16.1.2
R2(config)#ip route 10.1.0.0 255.255.255.0 172.16.1.1
The above is directing LAN traffic for both sites on the 10.1.0.0/10.2.0.0 through the overlay GRE tunnel. However, packets that are not destined for this will still go through the standard default route. Meaning they will not go through the overlay. In our topology we can see this when Carol makes a TCP 443 web request to the server in the DC.
Comments