On Di, 2014-02-18 at 12:03 +0200, Laine Stump wrote:> You *really* should upgrade to a newer libvirt.I know that version 0.9.8 is very old. But to be honest i tried to avoid upgrading and compiling a newer version since i don't know if it has any effects on running VMs (but i haven't checked this yet). Its a production server and i did not want to interrupt any services running on these VMs. However, i'm afraid that upgrading might be the only option if i want to avoid setting up iptables manually.> Those previous two rules are the ones added when you specify a forward > dev. You don't need to do that - I would recommend removing the > "dev='eth0'" from the <forward> element of the network, along with the > "<interface dev='eth0'/>" subelement. This won't change operation at > all, it will just make things slightly less confusing and misleading.Thank you for pointing this out. I just did that. And after a reboot everything is still working as expected (yes. I just DID interrupt the services running in my VMs. So i guess i could even upgrade to a newer libvirt =) )> > > > -A libvirt-out -m physdev --physdev-out vnetX -g FO-vnetX > > > > which should trigger the following rules: > > > > -A FO-vnetX -p all -m state --state ESTABLISHED -j ACCEPT > > -A FO-vnetX -p tcp --dport 22 -m state --state NEW -j ACCEPT > > > > But this actually never happens. The FO-vnetX Chain never sees any packets and my syslog says: > > > > xt_physdev: using --physdev-out in the OUTPUT, FORWARD and POSTROUTING chains for non-bridged traffic is not supported anymore> That somehow sounded familiar, so I looked it up in the git history and > found this: > > http://libvirt.org/git/?p=libvirt.git;a=commit;h=65fb9d49cc9caae210977934b53d87e56429407b > > That patch was included in libvirt-1.0.2, just about a year ago.After reading that i remove the following iptables rule: iptables -D libvirt-out -m physdev --physdev-out vnetX -g FO-vnetX and manually added this rule: (the patch said that adding an extra argument (--physdev-is-bridged) is needed for rules like this): iptables -A libvirt-out -m physdev --physdev-is-bridged --physdev-out vnetX -g FO-vnetX Indeed this prevents my syslog from being spammed with the mentioned warning. However, this did not fix the problem. This rule never matches anything, and thus the FO-vnetX Chain never sees any packets. Using a testing rule like: iptables -A libvirt-out -d 1.2.3.70 -g FO-vnetX made everything work as expected. Well, this is definitely not the way it is expected to work since it does not do any "bridge port" matching. This all makes me think its not a libvirt specific problem and updating to a newer version will not fix my problem. Maybe there is somebody out there using a setup like mine and can show me the rules that a getting created with a newer version of libvirt? So long, thank you for all the usefull information! Kind regards, Sebastian
On 02/18/2014 06:31 AM, h0rst wrote:> On Di, 2014-02-18 at 12:03 +0200, Laine Stump wrote: > >> You *really* should upgrade to a newer libvirt. > > I know that version 0.9.8 is very old. But to be honest i tried to avoid upgrading and compiling a newer version since i don't know if it has any effects on running VMs (but i haven't checked this yet).There should be no problem upgrading to a newer libvirt. We take great pains to ensure that a newer version of libvirt can be reloaded and gracefully understand the XML recorded by older versions, with no loss to running VMs. While there have been bugs on this front, they get caught and patched quickly, so by updating to something like the latest Fedora stable build (currently 1.1.3.3), you are even more likely to avoid these sorts of problems when compared to upgrading all the way to the master branch of libvirt. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Di, 2014-02-18 at 16:06 -0700, Eric Blake wrote:> There should be no problem upgrading to a newer libvirt. We take great > pains to ensure that a newer version of libvirt can be reloaded and > gracefully understand the XML recorded by older versions, with no loss > to running VMs. While there have been bugs on this front, they get > caught and patched quickly, so by updating to something like the latest > Fedora stable build (currently 1.1.3.3), you are even more likely to > avoid these sorts of problems when compared to upgrading all the way to > the master branch of libvirt.After i forced myself to upgrade and compile a newer version of libvirt, (to be precise, i wanted to upgrade to the latest libvirt-1.2.1.tar.gz) i ran into some problems. I compiled libvirt with the following options: As a reminder, i'm running ""good"" old Ubuntu 12.04: ./configure --with-lxc --with-storage-lvm --prefix=/usr --localstatedir=/var --sysconfdir=/etc Compiling and installing worked perfectly after installing all missing dependencies. At first everything looked fine and all created networks and domains where still running. To be sure everything would survive a system restart, i rebooted. Thats when everything (or something) went wrong. After starting libvirtd i got the following error:>>> error: Failed to start network hetzner-subnet-v4 >>> error: unsupported configuration: Publicly routable address 1.2.3.65 is prohibited. >>> The version of dnsmasq on this host (2.59) doesn't support the bind-dynamic option >>> or use SO_BINDTODEVICE on listening sockets, one of which is required for safe >>> operation on a publicly routable subnet (see CVE-2012-3411). You must either upgrade >>> dnsmasq, or use a private/local subnet range for this network (as described in RFC1918/RFC3484/RFC4193).Since no VM was running at this point (because of the missing networks), i decided to quickly update to a newer version of DNSMASQ (2.68) and installed this to "/usr/local/sbin" and linked it to "/usr/sbin/dnsmasq" after removing the distribution specific packages. When i tried to start the hetzner-subnet-v4 network, i got the following error:>>> 2014-02-19 14:11:58.636+0000: 7075: error : virCommandWait:2376 : internal error: Child process (LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOME=/root USER=root LOGNAME=root /usr/sbin/dnsmasq --version) unexpected exit status 1: libvirt: error : cannot execute binary /usr/sbin/dnsmasq: Permission denied>>> 2014-02-19 14:11:58.636+0000: 7075: error : dnsmasqCapsRefreshInternal:747 : failed to run '/usr/sbin/dnsmasq --version': : Success >>> error: Failed to start network hetzner-subnet-v4 >>> error: failed to run '/usr/sbin/dnsmasq --version': : SuccessHowever, running dnsmasq manually worked. Since that was the moment the first phonecalls started because users could not access their services on the VMs i quickly reverted everything to its previous state to get everything up and running again. I dont have any testing server, so i could not play around with it anymore (better to say not right now. I might have to wait until everyone is sleeping ;)). Does libvirt has any problems when accessing a softlink instead of a binary? Poorly that possibility came into my mind after reverting back to its original state! I'm really sorry to spam you guys with all my problems ;) Kind regards, Sebastian