Introduction

I've used VMware workstation for virtualization. You can use Virtual Box and follow this steps. Just interface name will be different.
Also, ubuntu server 22.04 iso is used for creating 3 virtual machines - client, router, server.

Create Network

• vmnet8: NAT - 192.168.22.0/24 - DHCP (for internet connection)

• vmnet2: Host-only - 10.10.1.0/24

• vmnet3: Host-only - 10.10.2.0/24

Building Router

Create a VM using ubuntu server 22.04 iso image.
It will use the following configurations:

• RAM: 2048 MB

• Processors: 1

• Network Adapter 1 → vmnet8

• Network Adapter 2 → vmnet2

• Network Adapter 3 → vmnet3

Setup the configuration of these 3 interface in /etc/netplan/*.yaml file.

network:
    ethernets:
        ens33:
            dhcp4: true
        ens36:
            addresses: [10.10.1.1/24]
        ens37:
            addresses: [10.10.2.1/24]
    version: 2

The above configuration:

1. Enables the DHCP protocol on ens33 interface. IP and gateway address will be set dynamically.

2. Sets IP address of ens36 interface.

3. Sets IP address of ens37 interface.

Change permission: sudo chmod 600 *.yaml
Apply the change: sudo netplan apply

Stop and remove UFW firewall.

sudo systemctl stop ufw
sudo apt purge ufw

Building Client

Clone the router VM. Go to settings and delete adapter 1, 3.

Configure the interface using netplan. in /etc/netplan/*.yaml file.

network:
    ethernets:
        ens36:
            addresses: [10.10.1.10/24]
            routes:
                - to: default
                  via: 10.10.1.1
            nameservers:
        addresses:
            - 8.8.8.8
            - 8.8.4.4
    version: 2

The above configuration:

1. Sets device IP address (10.10.1.10) and Netmask (/24).

2. Sets any destination IP address route except this network will go to specific gateway (10.10.1.1).

Apply the change: sudo netplan apply

Building Server

Clone the router VM. Go to settings and delete adapter 1, 2.

Configure the interface in /etc/netplan/*.yaml file.

network:
    ethernets:
        ens37:
            addresses: [10.10.2.20/24]
            routes:
                - to: default
                  via: 10.10.2.1
            nameservers:
        addresses:
            - 8.8.8.8
            - 8.8.4.4
    version: 2

Apply the change: sudo netplan apply

Now both client and server can communicate with the router.

Client (or Server) VM can't ping the Router VM?

Ping client (or server) from the router.
This will invoke the ARP request to get the MAC address of client (or server).

Enable packet forwarding

Router need to forward packet from one interface to another. Thant's why we need to enable packet forwarding.

sudo echo 1 > /proc/sys/net/ipv4/ip_forward

Source NAT for local network

From the configured infrastructure, we know that two host-only network is in 10.10.1.0/24 & 10.10.2.0/24 range. Devices connected in these network is configured with 10.10.1.1 & 10.10.2.1 IP address respectively as Gateway IP.

(we don't need to configure MAC address manually. All interface will get the MAC address through ARP broadcast, if they don't have it in cache.)

The router has three network interfaces; ens36 & ens37 for two different host-only network and ens33 for internet-facing connection.

Now your packet from client (or server) can go to routers appropriate interface. If the packet goes from one network to another, the gateway would not recognize the source IP. It will send ARP request to the network to get MAC address of the source IP. If it doesn't get response, it will drop it.

So, to send the packet of different network, we are going to change the source IP address with interfaces own IP address.

We will run the following two rules in router VM to do IP Masquerading when the packet is coming form 10.10.1.0/24 and 10.10.2.0/24 network.

sudo iptables -t nat -s 10.10.1.0/24 -A POSTROUTING -j MASQUERADE

sudo iptables -t nat -s 10.10.2.0/24 -A POSTROUTING -j MASQUERADE

Now, say a device in local network with IP address 10.10.1.10 wants to access a website on the internet. When the device sends a packet to the gateway, the iptables rule kicks in.

The source IP address (10.10.1.10) of the outgoing packet is dynamically replaced with the routers private IP address of the ens33 interface.

This is called IP Masquerading. In this process, nothing is changed except the source IP.

Tracing IP on different interface

I have used tcpdump to check the source and destination IP address of a packet in every interface.

The following table shows when a packet is going form client machine to server machine:

The following table shows when a packet is going from server machine to internet:

How packet is moving from one interface to other

When a packet comes, linux kernel checks the routing table to determine the appropriate interface for the packet.

If the packet was destined for the internet, the routing table directs the packet to the ens33 interface. After going to that interface, in POSTROUTING chain, the source IP is changed to current interface's IP.

If the packet was destined for the server VM, the routing table would direct the packet to ens37 interface. Similar to above, the source IP would change.