libvirt users - Aug 2013 - Strange connectivity issues with bridged networking and masquerade

Hi all,

I'm currently in the process of building a 2-Node libvirt/KVM Cluster
and ran into some issues regarding the network connectivity of our
virtual machines.

Our setup seemed to work fine, we were able to browse to Google and our
own company website and some others from within the VM. Then we tried
microsoft.com to download some Windows iso images from MS Partner
Network. The page started to load, but only a few elements became
visible - then it sticked to: "Loading data from
microsoft.com" ...forever. A few other examples that do not work are:

* www.opera.com
* www.amazon.com
* www.speedtest.net

All of these pages load without any problem, when I access them from my
laptop or even with Firefox via X-Forwarding launched directly on the
hypervisor system. From within the VMs they just refuse to finish
loading. The only thing those pages have in common, as far as I can see,
is that they heavily utilize CDNs like Amazon Cloudfront or Akamai.

The idea behind our setup is, that all virtual machines communicate on
the 192.168.3.0/24 network. The nodes have a VLAN connection on eth1. To
allow connections between VMs on different hosts, we created the bridge
device br1 with eth1 attached and added the VMs to it. eth0 provides
internet access with xx.xx.220.0 as additional public failover ip.

We added 192.168.3.254 as additional IP to one of the node's br1 device
to use it as the default gateway for the VMs. This IP can be migrated
between the nodes.

Our setup looks like this:
                  ____________
                 /            \
                (   Internet   )
                 \____________/
                   /        \
Node1:             |        |     Node2:        
                   |        |
xx.xx.217.8     (eth0)    (eth0)    xx.xx.217.10
xx.xx.220.0        \
                 {Masq.}
                   /
                (eth1)----(eth1)
                   |        |
192.168.3.1     [br1 ]    [br1 ]     192.168.3.2
192.168.3.254      |        |              
                   |        |
192.168.3.50   (vnet0)    (vnet0)   192.168.3.75

----------
  iptables looks like this:
  root@vm01:~# iptables -S
  -P INPUT ACCEPT
  -P FORWARD ACCEPT
  -P OUTPUT ACCEPT
  -A FORWARD -d 192.168.3.0/24 -o br1 -m state --state
RELATED,ESTABLISHED
  -j ACCEPT
  -A FORWARD -s 192.168.3.0/24 -i br1 -j ACCEPT
  -A FORWARD -i br1 -o br1 -j ACCEPT
  -A FORWARD -i eth0 -o eth0 -j ACCEPT
  root@vm01:~# iptables -S -t nat
  -P PREROUTING ACCEPT
  -P INPUT ACCEPT
  -P OUTPUT ACCEPT
  -P POSTROUTING ACCEPT
  -A POSTROUTING -s 192.168.3.0/24 ! -d 192.168.3.0/24 -j MASQUERADE
  -A POSTROUTING ! -s 192.168.3.0/24 -d 192.168.3.0/24 -j MASQUERADE
---------

Some additional information that might be helpful:
  root@vm01:~# virsh version
  Compiled against library: libvir 0.9.12
  Using library: libvir 0.9.12
  Using API: QEMU 0.9.12
  Running hypervisor: QEMU 1.1.2
---------
  root@vm01:~# uname -a
  Linux vm01.cluster 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64
GNU/Linux
---------

I don't know if it's really libvirt-related but perhaps someone here has
an idea what to try. Any advice on this is really appreciated, as  I am
at my wits' end. Thank you in advance... :)

Kind regards
Kolja Scheffler

On 23/08/13 15:02, Kolja Scheffler wrote:
> I don't know if it's really libvirt-related but perhaps someone
> here has an idea what to try. Any advice on this is really
> appreciated, as  I am at my wits' end. Thank you in advance... :)
I had issues with kvm and linux-bridge and the 3.x kernel in Debian,
can you try the 2.6 kernel in oldstable/security-updates. People
recommended me to use openvswitch instead of linux-bridge-utils with a
3.x kernel. Don't know if this will help you, but you can try it.

Kind regards,

Jelle

> People recommended me to use openvswitch instead of linux-bridge-utils with
a
> 3.x kernel.
Thanks for the reply. I think I'll give openvswitch a try and report
back the results.
> _______________________________________________
> libvirt-users mailing list
> libvirt-users@redhat.com
> https://www.redhat.com/mailman/listinfo/libvirt-users