The Fundamentals of Geographically Dispersed Communication

This is a followup on The Fundamentals of Separate LAN Communication

In the previous blog of this series we discussed how a router which is physically and logically connected to two different networks, how it will facilitate the communication from Alice in the 10.10.10.0/24 network to Bob in 10.10.20.0/24. We know that ARP does not cross the Router's boundary. What does this mean? An ARP Broadcast from Alice will not be forward to Bob's network. It does however have the capability to see an ARP message, check the Source IP and Target IP and check its routing table to create its own ARP requests and replies.

However, what if in the case of where the network is not directly connected to the router? What if there is multiple routers in between the source and destination? What if Bob has moved to a different location? What if Alice needs to communicate to other people like david?

Topology Overview

Here we have 3 LANs - UK, US and South Korea.

1. UK LAN - This includes Alice in the 10.10.10.0/24 Network as well as its default gateway R1.
2. US LAN - This includes Bob in the 10.10.20.0/24 Network as well as its default gateway R2.
3. South Korea LAN - This includes David in the 10.10.20.0/24 Network as well as its default gatweay R3.

I have also added some internet routers as just hops. I have given them IPs in the 203.0.x.x range just to indicate that they are internet routers and not part of the Private IP Ranges. As a reminder private IP Ranges are 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16.

The point of this is to show how we can communicate when router is not directly connected to our destination network without going in-depth into routing.

How these hosts communicate

Alice to R1

Once again we will start with Alice who wants to communicate with David. Alice will generate the IP Packet let's assume that she is pinging David on 10.10.30.150. The IP Packet will look like the below:

We know by now that if we are using ethernet wiring, that this IP packet cannot be sent on the wire as is, it needs to be encapsulated into a L2 Header. I will skip the explanation of why, refer to the previous blogs in the series. All we need to understand is the frame will be committed into the memory of the host. The host will know that this destination is not in its own network, and it is configured with a default gateway so Alice will send an ARP Request towards R1 as we see below:

As we know, for the ARP request to go on the Ethernet Segment/Wire it will need to be encapsulated into a L2 Ethernet frame as we see below:

The switch receives this broadcast, learns Alice's MAC was learnt on Gi0/1 and then broadcasts it out of all ports except the one it received it on. This means it reaches R1. I wont go through the ARP Reply in detail, but in essence R1 gets the frame, de-encapsulates the L2 header and then looks at the ARP packet. It sees Target IP is for itself and then forwards the reply back to Alice.

R1 to R2

We have covered this already. What is different is how when Alice sends the ICMP message now to R1 (encapsulated into the complete L2 Ethernet Frame) the Router sees after de-encapsulating the frame, that the L3 header the destination IP is destined to 10.10.30.150.

The Router will look into its routing table and this routing table will tell it how to reach 10.10.30.150 out of what interface/next hop. How does it learn this information? Via a dynamic routing protocol. We wont be covering this but I will show R1's routing table below:

The source and destination IP Address WILL NEVER change (until the destination receives this packet and responds) however the MAC Address at every hop WILL change. We run into the same issue because R1 has to re-encapsulate this IP Packet into an L2 Frame and it knows to reach 10.10.30.150 it must go via R2. But what is R2's MAC Address? It doesnt know. So it caches this incomplete L2 Frame in memory and once again repeats the process and does an ARP Request however this time it will look like the below:

As we understand by now, the Internet1 device will de-encapsulate this frame and check the ARP request and see Target IP is for itself and then send back a Unicast ARP Reply message back to R1.

R1 then sends the completed L2 Frame that includes the L3 Header/ICMP message to the Internet1 Router.

Internet1 to Internet2 to R3

Internet1 de-encapsulates the L2 Ethernet Frame and removes the L2 header and sees the L3 header which includes the source and destination IP Addresses. Below is Internet1's Routing Table:

Internet1 Router knows it has to forward this L3 packet out of its Gi0/3 interface. We know by now it has to encapsulate this into a L2 Frame and produce an ARP request. I wont go over it again.

This process repeats all the way through from Internet1 to Internet2 to R3

R3 to David

When R3 receives the Completed L2 frame from Internet 2 it will de-encapsulate the L2 Ethernet Frame and looks into the L3 Header and looks in its routing table:

This time the Router sees it has a connected route for the end destination so it encapsulates it into an incomplete L2 Ethernet Frame and stores it in memory. It then sends an ARP for David, the switch receives it since its a broadcast frame, the switch floods it out of all ports except the one it received it on, so David gets it.

David then sends an ARP reply and eventually R3 sends the completed L2 Frame to David.

David to Alice

David receives the complete L2 Ethernet Frame and de-encapsulates it and removes the L2 header and looks into the L3 IP Packet and sees the destination is for itself. The ICMP Echo written in the protocol itself requires a ICMP Reply ping to be sent.

So David will create this reply packet. Whats important to understand is that the source IP and destination IP will be reversed it will look like the below:

David encapsulates it into an L2 Frame however NO ARP is required. Why? Because R3 had already sent an ARP Request earlier and David has learnt R3's MAC Address. The switch will get the L2 Ethernet Frame and forward to R3.

Now from R3 all the way back to Alice ARP should not happen again because each MAC along this path should be known in their respective ARP Cache. ARP Request and replies have already been done. The Full communication is now complete.