Why VXLAN cannot be integrated into original packet
You may be wondering like I was why VXLAN could not just be integrated into the original IP packet which was sent by the host Carol? The reason for this is because if we change the software code of IP, then every single network device in the path of dealing with this new IP packet will need the updated IP code.
There are many devices like routers which may not understand VXLAN information without a software update if we made changes to IP packet. So instead only the devices participating in the SDA fabric (we will discuss later) the Edge, intermediate and core switches need VXLAN capability.
Standard IP packet vs VXLAN Packet
Read these from right to left.
Standard IP Packet:
- The Payload here would be ICMP traffic that carol is initiating to David.
- The IP header includes information about source and destination. This is not the loopback, but the actual end host IPs.
- And the ethernet header actually transmits the packet on the wire using MAC addresses. This is the source & destination MACs that the switches have learnt from the control plane.
VXLAN packet:
- VXLAN includes the VNI - The Virtual Network Identifier which includes VLAN information. VLANs are mapped to VNI. The VNI field is 24 bits long
Warning
Something to always remember is that VLANs are mapped to VNIs yes, but a VNI does not need to be the same number as the VLAN. What do I mean?
For example:
Carol = VLAN 10. VXLAN maps VLAN 10 to VNI 500
David = VLAN 50. VXLAN maps VLAN 50 to VNI 500
Still in the same IP network (possible in SD Access)
What this means is when SW1 encapsulates the packet it wont put the VLAN 10 or 50 number, it will put the VNI 500 number. The VLAN number is only locally significant on the switch
So if Carol pings David, SW1 will encapsulate the packet, Query LISP and do all its checks, and then will forward packet to David. SW2 will de-encapsulate and see that the VNI value is 500. On SW2 VNI 500 is mapped to VLAN 20 so it knows to forward out of VLAN 500.
- The VNI number is global but the VLAN number is locally significant on the switch.
- The UDP header is used for Load balancing based on UDP SRC-Port. If we have ECMP going on when sending out a VXLAN packet it will load balance. By default the source and destination of the packet will always be the same when any host behind SW1 attempts to send a VXLAN packet. Why? Because we are using the loopback addresses remember as the OSA and ODA. Source & Destination IP cannot be used.
- The UDP header is encapsulated further encapsulated into the OSA and ODA. This in our examples so far has been SRC 10.10.10.1 and DST 10.10.10.2.
- The Ethernet Header helps transmit it over the L3 links, SRC MAC will be SW1 and we get the DST MAC from the control plane node.
- Technically from IP to Application Payload it is all actually encapsulated into Ethernet and the rest is payload.
Note
For clarification on the UDP. Remember when we send an ARP request from a client it reaches the switch but ARP cannot cross L3. So the SW requests from the control plane the IP of the destination host. I wont go into mechanisms of LISP, but it basically provides the IP address of the remote switch that the endpoint is connected to. And this IP will always be the loopback address. Why the loopback? Because every node in the SDA fabric is peering on loopback addresses.
What does this have to do with load balancing? Well if a host Carol and Sam on SW1 attempt to communicate with David who is connected on SW2. How can we load balance these two flows if they are both going to be using the same SRC IP of the Outer Source Address? We cant. That's why UDP is used.
Below is a fantastic diagram that shows an overview of all of this
- VTEP is just "VXLAN Tunnel Endpoint" This is the loopback we have created on the access switches.
Recommended materials
SD-Access Data Plane part 2 - Highly Recommend
Comments