My intend is to test and understand the reside-on-chassis and redirect-type options on DVR ports.
Topology
I have used GNS3 for testing OVN setup;
- All servers are using Fedora-Cloud-Base-38-1.6
- All servers have 3 interface;
- One interface for management access
- One interface for OVN control packets and overlay tunnel interface.
- One interface for external connection, simulation TOR switch connection for various purposes used on tests.
- I have not used gw2, it can be used for HA tests.
Preparing Servers
1. Change hostnames
2. On all nodes
sudo setenforce 0
sudo sed -i ‘s/^SELINUX=.*/SELINUX=permissive/g’ /etc/selinux/config
sudo dnf update -y
#Just for OVN Central node
sudo yum install ovn-central -y
#For all OVN nodes
sudo yum install ovn -y
sudo yum install ovn-host -y
sudo dnf install network-scripts -y
sudo systemctl disable NetworkManager
sudo systemctl stop NetworkManager
sudo systemctl enable network
sudo systemctl start network
3. Configure interfaces
– Management interface
/etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=”eth0″
BOOTPROTO=”dhcp”
ONBOOT=”yes”
TYPE=“Ethernet”
– OVN interface
/etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=”eth1″
BOOTPROTO=”static”
ONBOOT=”yes”
TYPE=”Ethernet”
PREFIX=24
#For Central
IPADDR=192.168.150.100
#For worker1
IPADDR=192.168.150.101
#For worker2
IPADDR=192.168.150.102
#For gw1
IPADDR=192.168.150.98
#For gw2
IPADDR=192.168.150.99
4. Restart Machines
Logical Topology
General logical topology used during tests.
OVN Nodes Setup
You have to reconfigure after restarting servers. There may be a way to make this persistent which is not my focus.
1. Central node configuration
hostname=$(hostname)
/usr/share/openvswitch/scripts/ovs-ctl start –system-id=$hostname
/usr/share/ovn/scripts/ovn-ctl start_ovsdb –db-nb-create-insecure-remote=yes –db-sb-create-insecure-remote=yes
/usr/share/ovn/scripts/ovn-ctl start_northd
2. Worker 1
hostname=$(hostname)
/usr/share/openvswitch/scripts/ovs-ctl start –system-id=$hostname
/usr/share/ovn/scripts/ovn-ctl start_controller
ovs-vsctl set open . external-ids:ovn-bridge=br-int
ovs-vsctl set open . external-ids:ovn-remote=tcp:192.168.150.100:6642
ovs-vsctl set open . external-ids:ovn-encap-type=vxlan
ovs-vsctl set open . external-ids:ovn-encap-ip=192.168.150.101
3. Worker 2
hostname=$(hostname)
/usr/share/openvswitch/scripts/ovs-ctl start –system-id=$hostname
/usr/share/ovn/scripts/ovn-ctl start_controller
ovs-vsctl set open . external-ids:ovn-bridge=br-int
ovs-vsctl set open . external-ids:ovn-remote=tcp:192.168.150.100:6642
ovs-vsctl set open . external-ids:ovn-encap-type=vxlan
ovs-vsctl set open . external-ids:ovn-encap-ip=192.168.150.102
4. Gw1
hostname=$(hostname)
/usr/share/openvswitch/scripts/ovs-ctl start –system-id=$hostname
/usr/share/ovn/scripts/ovn-ctl start_controller
ovs-vsctl set open . external-ids:ovn-bridge=br-int
ovs-vsctl set open . external-ids:ovn-remote=tcp:192.168.150.100:6642
ovs-vsctl set open . external-ids:ovn-encap-type=vxlan
ovs-vsctl set open . external-ids:ovn-encap-ip=192.168.150.98
ovs-vsctl set open . external-ids:ovn-cms-options=”enable-chassis-as-gw”
Creating Fake VM for our tests
Again there may be a way to make this persistent which is not my focus.
1.Worker1
#fake vm for worker1
ovs-vsctl add-port br-int vm1 — set Interface vm1 type=internal — set Interface vm1 external_ids:iface-id=vm1
ip netns add vm1
ip link set vm1 netns vm1
ip netns exec vm1 ip link set vm1 address 40:44:00:00:10:01
ip netns exec vm1 ip addr add 10.10.10.101/24 dev vm1
ip netns exec vm1 ip link set vm1 up
ip netns exec vm1 ip route add default via 10.10.10.1
ovs-vsctl add-port br-int vm3 — set Interface vm3 type=internal — set Interface vm3 external_ids:iface-id=vm3
ip netns add vm3
ip link set vm3 netns vm3
ip netns exec vm3 ip link set vm3 address 40:44:00:00:20:03
ip netns exec vm3 ip addr add 20.20.20.103/24 dev vm3
ip netns exec vm3 ip link set vm3 up
ip netns exec vm3 ip route add default via 20.20.20.1
2. Worker2
#fake vm
ovs-vsctl add-port br-int vm2 — set Interface vm2 type=internal — set Interface vm2 external_ids:iface-id=vm2
ip netns add vm2
ip link set vm2 netns vm2
ip netns exec vm2 ip link set vm2 address 40:44:00:00:10:02
ip netns exec vm2 ip addr add 10.10.10.102/24 dev vm2
ip netns exec vm2 ip link set vm2 up
ip netns exec vm2 ip route add default via 10.10.10.1
ovs-vsctl add-port br-int vm4 — set Interface vm4 type=internal — set Interface vm4 external_ids:iface-id=vm4
ip netns add vm4
ip link set vm4 netns vm4
ip netns exec vm4 ip link set vm4 address 40:44:00:00:20:04
ip netns exec vm4 ip addr add 20.20.20.104/24 dev vm4
ip netns exec vm4 ip link set vm4 up
ip netns exec vm4 ip route add default via 20.20.20.1
Tests
Test 1 DVR Basic Setup
On OVN central create the networks, router and e.t.c.
#blue network
ovn-nbctl ls-add network-blue
ovn-nbctl lsp-add network-blue vm1
ovn-nbctl lsp-add network-blue vm2
ovn-nbctl lsp-set-addresses vm1 “40:44:00:00:10:01 10.10.10.101”
ovn-nbctl lsp-set-addresses vm2 “40:44:00:00:10:02 10.10.10.102”
#green network
ovn-nbctl ls-add network-green
ovn-nbctl lsp-add network-green vm3
ovn-nbctl lsp-add network-green vm4
ovn-nbctl lsp-set-addresses vm3 “40:44:00:00:20:03 20.20.20.103”
ovn-nbctl lsp-set-addresses vm4 “40:44:00:00:20:04 20.20.20.104”
#router
ovn-nbctl lr-add router1
ovn-nbctl lrp-add router1 router1-netblue-int 40:44:00:00:10:EE 10.10.10.1/24
ovn-nbctl lsp-add network-blue netblue-router1-int
ovn-nbctl lsp-set-type netblue-router1-int router
ovn-nbctl lsp-set-addresses netblue-router1-int router
ovn-nbctl lsp-set-options netblue-router1-int router-port=router1-netblue-int
ovn-nbctl lrp-add router1 router1-netgreen-int 40:44:00:00:20:EE 20.20.20.1/24
ovn-nbctl lsp-add network-green netgreen-router1-int
ovn-nbctl lsp-set-type netgreen-router1-int router
ovn-nbctl lsp-set-addresses netgreen-router1-int router
ovn-nbctl lsp-set-options netgreen-router1-int router-port=router1-netgreen-int
As it is seen, we do not need gateway chassis for E-W traffic. E-W traffic is tunneled to the destination OVN host. Its first routed to the destination network on the host and switched to the destination network. If the destination is on a remote host its tunneled to that host.
Test 2 Adding external network
Adding external network to our router. We will just ping the connected interface of our external host. Beside that will require adding a default route on the OVN..
On gw1 adding external net to the ovs
ip link set dev eth2 up
ovs-vsctl –may-exist add-br br-ex
ovs-vsctl add-port br-ex eth2
ovs-vsctl br-set-external-id br-ex bridge-id br-ex
ovs-vsctl set open . external-ids:ovn-bridge-mappings=external:br-ex
On ovn-central configuring router
ovn-nbctl ls-add public
ovn-nbctl lsp-add public public-localnet
ovn-nbctl lsp-set-type public-localnet localnet
ovn-nbctl lsp-set-addresses public-localnet unknown
ovn-nbctl lsp-set-options public-localnet network_name=external
ovn-nbctl set Logical_Switch_Port public-localnet tag=254
ovn-nbctl lrp-add router1 router1-public 40:44:EE:EE:EE:EE 172.24.4.1/24
ovn-nbctl lsp-add public public-router1
ovn-nbctl lsp-set-type public-router1 router
ovn-nbctl lsp-set-addresses public-router1 router
ovn-nbctl lsp-set-options public-router1 router-port=router1-public
ovn-nbctl lrp-set-gateway-chassis router1-public gw1
This can be solved by adding any kind of NAT for the source or adding a return path on the external network.
Test 3 using localnet
We are now extending our tenant networks, blue and green to the external net using vlans. This requires configuring external network for those vlans.
# workers do not need external net, they just need tenant net mapping
ip link set dev eth2 up
ovs-vsctl –may-exist add-br br-ex
ovs-vsctl add-port br-ex eth2
ovs-vsctl br-set-external-id br-ex bridge-id br-ex
ovs-vsctl set open . external-ids:ovn-bridge-mappings=tenant:br-ex
#gw need need external and tenant net
ovs-vsctl set open . external-ids:ovn-bridge-mappings=external:br-ex,tenant:br-ex
#each switch need to have a localnet for external network
ovn-nbctl lsp-add network-blue netblue-localnet
ovn-nbctl lsp-set-type netblue-localnet localnet
ovn-nbctl lsp-set-addresses netblue-localnet unknown
ovn-nbctl lsp-set-options netblue-localnet network_name=tenant
ovn-nbctl set Logical_Switch_Port netblue-localnet tag=10
ovn-nbctl lsp-add network-green netgreen-localnet
ovn-nbctl lsp-set-type netgreen-localnet localnet
ovn-nbctl lsp-set-addresses netgreen-localnet unknown
ovn-nbctl lsp-set-options netgreen-localnet network_name=tenant
ovn-nbctl set Logical_Switch_Port netgreen-localnet tag=20
As we are now bind our tenant networks to the external network all communication between those networks will use external vlans. The main difference is for external traffic.
Now the external traffic will first of the gateway node over tunnel interface, then it is all over external net.
Test 4 Using Reside on Chassis solving test 3 external traffic tunneling
ovn-nbctl lrp-set-options router1-netblue-int reside-on-redirect-chassis=true
ovn-nbctl lrp-set-options router1-netgreen-int reside-on-redirect-chassis=true
Test-5 Using redirect-type bridge
First clear all options
ovn-nbctl lrp-set-options router1-netgreen-int reside-on-redirect-chassis=false
ovn-nbctl lrp-set-options router1-netblue-int reside-on-redirect-chassis=false
ovn-nbctl lsp-del netblue-localnet
ovn-nbctl lsp-del netgreen-localnet
On worker 1, worker 2 and worker 3
ovs-vsctl set open . external-ids:ovn-bridge-mappings=external:br-ex
Check with
ovs-vsctl get open . external-ids:ovn-bridge-mappings
Worker 1
ovs-vsctl set open . external-ids:ovn-chassis-mac-mappings=”external:40:44:EE:EE:EE:01″
Worker 2
ovs-vsctl set open . external-ids:ovn-chassis-mac-mappings=”external:40:44:EE:EE:EE:02″
Gw1
ovs-vsctl set open . external-ids:ovn-chassis-mac-mappings=”external:40:44:EE:EE:EE:98″
