Open vSwitch with KVM/libvirt in Debian Stretch

Just switched from a plain linux bridge to Open vSwitch.

Installation & Initial Setup

The installation is as simple as issuing apt-get install openvswitch-switch on the console. The documentation states that e.g.

ex 1: A standalone bridge.
 
allow-ovs br0
iface br0 inet static
    address 192.168.1.1
    netmask 255.255.255.0
    ovs_type OVSBridge
 
ex 2: A bridge with one port.
 
allow-ovs br0
iface br0 inet dhcp
    ovs_type OVSBridge
    ovs_ports eth0
 
allow-br0 eth0
iface eth0 inet manual
    ovs_bridge br0
    ovs_type OVSPort

Should work. So I configured:

allow-ovs ovsbr0
iface ovsbr0 inet static
    address x.x.x.200
    netmask 255.255.255.192
    broadcast x.x.x.255
    gateway x.x.x.193
    ovs_type OVSBridge
    ovs_ports eth1
 
allow-ovsbr0
iface eth1 inet manual
    ovs_bridge ovsbr0
    ovs_type OVSPort

However, it seems there are some issues on Debian with the start scripts. Without any auto clause in /etc/network/interfaces neither ovsbr0 nor my primary network device eth1 is started (I did wait like 10 minutes). My logfiles do not show anything special.

Searching a bit I stumbled upon the following bugreport openvswitch-switch: switch takes a very long time to start or fails without upstream’s SYSTEMCTL_SKIP_REDIRECT=yes. ovs-vsctl show does not show eth1. Adding eth1 manually using ovs-vsctl add-port ovsbr0 eth1 works, still no connectivity, though. Up’ing the link: ip link set eth1 up followed by ip link set ovsbr0 up worked. Finally I configured it as shown below, mind that this will make the whole boot-process delay by 5 minutes since networking/openvswitch depend on each other. However, that was the only way to get everything up and running automatically:

 allow-ovs ovsbr0
iface ovsbr0 inet static
    address x.x.x.200
    netmask 255.255.255.192
    broadcast x.x.x.255
    gateway x.x.x.193
    ovs_type OVSBridge
    ovs_ports eth1
    post-up /sbin/ip link set dev eth1 up
 
auto eth1
allow-ovsbr0 eth1
iface eth1 inet manual
    ovs_bridge ovsbr0
    ovs_type OVSPort
    post-up ip link set dev ovsbr0 up
    pre-down ip link set dev ovsbr0 down

And yes, I am aware that one shouldn’t use auto – Still that was the only way to get the devices up at boot.

Configuring KVM

Looks like my current setup did not need many changes, I just had to add <virtualport type='switchport'/> and replace vmbr0 with ovsbr0 in the interface configuration using virsh edit vmid. E.g:

    <interface type='bridge'>
      <mac address='xx:xx:xx:xx:xx:xx'/>
      <source bridge='ovsbr0'/>
      <virtualport type='openvswitch'/>
      <target dev='veth105'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

What does it look like?

root@janice:~# ovs-vsctl show
x-x-x-x-x
    Bridge "ovsbr0"
        Port "veth111"
            Interface "veth111"
        Port "eth1"
            Interface "eth1"
        Port "veth109"
            Interface "veth109"
        Port "veth106"
            Interface "veth106"
        Port "veth105"
            Interface "veth105"
        Port "veth113"
            Interface "veth113"
        Port "ovsbr0"
            Interface "ovsbr0"
                type: internal
        Port "veth112"
            Interface "veth112"
    ovs_version: "2.6.2"

4 Comments

  • Deucalion Dawson

    2. September 2019 at 18:15 Antworten

    Hello Jean-Michel,

    Nice topic, thanks for.
    I’m a rookie regarding the network configuration with ovs(openvswitch), so I ask you for a little indulgency and your help please.

    I have 2 proxmox hosts in cluster mode. each of the hosts has a single physical network card.

    my goal is to create :
    _a network for the 2 hosts proxmox physical hosts
    _a second one for vms only without use vlan if possible.

    Knowing that nothing is impossible in computers and especially in linux world, how would such a configuration with ovs look like ?

    thanks for your advice…

    • jean

      2. September 2019 at 19:07 Antworten

      Hi,

      first of all I’d advise against using Proxmox with just two hosts in „cluster mode“. Your very first problem is split-brain with a two-node-setup which is not easily solvable. I believe my example setup should work fine for what you’re doing: just configure the network manually and NOT using Proxmox. In Proxmox you’ll later just attach the specific bridge. While I do use Proxmox a lot at work, I never used it together with OpenVSwitch so I’m not entirely sure how to do that.

      Does that help?

  • Deucalion Dawson

    3. September 2019 at 10:37 Antworten

    Hi,

    Okay, I use proxmox just for testing purposes and I wanted to use ovs to separate :

    _ The company’s physical network (only my 2 proxmox hosts on those network for management)

    _ And another single network (e. g. internal) for vms communication between the 2 nodes of the cluster and unachievable from the company’s physical network

    Now, I have an idea thanks to your config and I don’t know if it will be ok but I thank you for your explanations.

    • jean

      4. September 2019 at 19:17 Antworten

      By the way, there are multiple options to solve what you want. One would be to work with two bridges, another one would be to just firewall the networks on the host-side. Another one would be to use openvswitch. Another one would involve routing and possibly a gre-tunnel and/or ipsec. Then there’s the possibility to use a hidden/transparent filtering VM (described that with ovs in my blog) or to just use an OPNSense as VM which would allow (just like ovs itself) to use vlans. Probably there’s some keyword in my text which does help you? 🙂

Post a Comment