David Raison
2016-Feb-11 20:11 UTC
[libvirt-users] Domain cannot talk to itself unless virbr0 is in promiscuous mode
Hi We have the following situation: A hypervisor using KVM-Qemu/libvirt with a single VM. The VM has the internal IP 192.168.122.151 on the default network pool using virbr0 on the hypervisor. We've set up netfilter rules to DNAT requests on one IP of the hypervisor to the VM. So, first of all, forwarding to the internal IP range is allowed:> $IPTABLES -I FORWARD -m state -d 192.168.122.0/24 --state > NEW,RELATED,ESTABLISHED -j ACCEPTThen we're forwarding a few ports from the hypervisor's public IP to the VM, imaps being one of them:> Chain PREROUTING (policy ACCEPT 2606 packets, 169K bytes) pkts bytes > target prot opt in out source destination 142 8048 DNAT tcp -- * * > 0.0.0.0/0 a.b.c.145 tcp dpt:993 to:192.168.122.151Since the VM is also doing SMTP, we need it to send out mail via a dedicated IPv4, which we achieve through the following SNAT rule:> Chain POSTROUTING (policy ACCEPT 1121 packets, 77556 bytes) pkts > bytes target prot opt in out source destination 81 6262 SNAT all -- * > * 192.168.122.151 0.0.0.0/0 to:a.b.c.145Now, on this mailserver, we're also running a roundcube instance. For authentication, roundcube connects to the imaps port of the local dovecot instance. In order for the SSL certificate check to succeed, it needs to do so using the actual domain name, and cannot use "localhost" or the loopback IP address. The problem is that, given the above iptables rules, connecting from inside the VM to itself via the hypervisor's external interface doesn't work. The connection simply times out, logging reveals they're not getting forwarded. We've added logging to iptables and testing in promisc mode vs. not-promisc: This is what gets logged when virbr0 is not in promisc mode:> Feb 11 19:12:31 hyper kernel: [199815.004207] prerouting imaps: > IN=virbr0 OUT= PHYSIN=vnet0 > MAC=fe:54:00:17:c8:2a:52:54:00:17:c8:2a:08:00 SRC=192.168.122.151 > DST=a.b.c.d.145 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=40689 DF > PROTO=TCP SPT=43460 DPT=993 WINDOW=29200 RES=0x00 SYN URGP=0And this when the device is in promisc mode:> Feb 11 19:14:33 hyper kernel: [199937.674189] prerouting imaps: > IN=virbr0 OUT= PHYSIN=vnet0 > MAC=fe:54:00:17:c8:2a:52:54:00:17:c8:2a:08:00 SRC=192.168.122.151 > DST=a.b.c.d.145 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=2244 DF PROTO=TCP > SPT=43461 DPT=993 WINDOW=29200 RES=0x00 SYN URGP=0 > Feb 11 19:14:33 hyper kernel: [199937.674221] forwarded imaps: IN=virbr0 > OUT=virbr0 PHYSIN=vnet0 MAC=52:54:00:17:c8:2a:52:54:00:17:c8:2a:08:00 > SRC=192.168.122.151 DST=192.168.122.151 LEN=60 TOS=0x10 PREC=0x00 TTL=63 > ID=2244 DF PROTO=TCP SPT=43461 DPT=993 WINDOW=29200 RES=0x00 SYN URGP=0 > Feb 11 19:14:33 hyper kernel: [199937.674766] forwarded imaps: IN=virbr0 > OUT=virbr0 PHYSIN=vnet0 MAC=52:54:00:17:c8:2a:52:54:00:17:c8:2a:08:00 > SRC=192.168.122.151 DST=192.168.122.151 LEN=52 TOS=0x10 PREC=0x00 TTL=63 > ID=2245 DF PROTO=TCP SPT=43461 DPT=993 WINDOW=229 RES=0x00 ACK URGP=0Another workaround we found is to disable iptables on the bridge interface.> root@hyper:~# echo 0 > /proc/sys/net/bridge/bridge-nf-call-iptablesBut I do not consider this a solution, merely a workaround. I do not understand why the forward rule mentioned above does not apply to the situation where IN and OUT are both virbr0? Thank you for any advice, David -- TenTwentyFour S.à r.l. W: www.tentwentyfour.lu T: +352 20 211 1024 F: +352 20 211 1023 9, avenue des Hauts-Fourneaux L-4362 Esch/Alzette
Maybe Matching Threads
- Samba4 + Bind-9.9.5: client update domain/IN denied for some hosts
- [Bridge] single briged network internet access problems
- Port forwarding from non-xenbridged external interface to xen-interface
- Strange TCP-Problem with GPL PV 0.8.8
- netfilter + vpn + how/why + etc...