libvirt users - Jun 2017 - Accepting RELATED, ESTABLISHED (TCP) connections into VM using Network Filters

Hi,

Over the past few days I've been trying to get a prototype working of a
stateful firewall for a Virtual Machine using Libvirt's network filters.

My goal is to replace the current custom Python/Java code in the Apache
CloudStack [0] project by Network Filters of Libvirt.

Both IPv4 and IPv6 should work, but I started off with IPv4 and I have issues
with accepting back RELATED,ESTABLISHED connections into the VM.

In the guest's XML I have this defined:

      <filterref filter='nwfilter-test'>
        <parameter name='IP' value='192.168.200.250'/>
        <parameter name='IPV6'
value='2001:db8:100:0:5054:ff:fe9c:6ce6'/>
        <parameter name='IPV6'
value='fe80::5054:ff:fe9c:6ce6'/>
      </filterref>

And the filter currently looks like this:

<filter name='nwfilter-test' chain='root'>
    <uuid>a2493284-9dd5-4c20-98b5-7e70745b53de</uuid>
    <!-- These are default build-in filters from libvirt and are mainly ipv4
only-->
    <filterref filter='no-mac-spoofing'/>
    <filterref filter='no-ip-spoofing'/>
    <filterref filter='no-arp-spoofing'/>
    <filterref filter='allow-dhcp'/>

    <!-- IPv4 Rules -->
    <rule action='accept' direction='in'
priority='100'>
      <all state='RELATED,ESTABLISHED'/>
    </rule>

    <rule action='return' direction='in'
priority='500'>
      <icmp/>
    </rule>

    <rule action='accept' direction='in'
priority='500'>
      <tcp dstportstart='22'/>
    </rule>

    <rule action='accept' direction='in'
priority='500'>
      <tcp dstportstart='80'/>
    </rule>

    <rule action='reject' direction='in'
priority='1000'>
      <all/>
    </rule>
</filter>

I can SSH into the VM and also visit the Webserver running on it. But going out
the VM results in issues:

root@nwfilter-test:~# telnet 109.72.92.155 80
Trying 109.72.92.155...
telnet: Unable to connect to remote host: Connection refused
root@nwfilter-test:~# 

I can however ping the same target:

root@nwfilter-test:~# ping -c 2 109.72.92.155
PING 109.72.92.155 (109.72.92.155) 56(84) bytes of data.
64 bytes from 109.72.92.155: icmp_seq=1 ttl=56 time=13.2 ms
64 bytes from 109.72.92.155: icmp_seq=2 ttl=56 time=14.1 ms

--- 109.72.92.155 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 13.225/13.703/14.182/0.492 ms
root@nwfilter-test:~#

Looking at iptables-save it seems like the right rules are programmed:

-A FI-vnet1 -p icmp -j RETURN
-A FI-vnet1 -p tcp -m tcp --sport 22 -m conntrack --ctstate ESTABLISHED -m
conntrack --ctdir REPLY -j RETURN
-A FI-vnet1 -p tcp -m tcp --sport 80 -m conntrack --ctstate ESTABLISHED -m
conntrack --ctdir REPLY -j RETURN
-A FI-vnet1 -j REJECT --reject-with icmp-port-unreachable
-A FO-vnet1 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FO-vnet1 -p icmp -j RETURN
-A FO-vnet1 -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -m
conntrack --ctdir ORIGINAL -j ACCEPT
-A FO-vnet1 -p tcp -m tcp --dport 80 -m conntrack --ctstate NEW,ESTABLISHED -m
conntrack --ctdir ORIGINAL -j ACCEPT
-A FO-vnet1 -j REJECT --reject-with icmp-port-unreachable
-A HI-vnet1 -p icmp -j RETURN
-A HI-vnet1 -p tcp -m tcp --sport 22 -m conntrack --ctstate ESTABLISHED -m
conntrack --ctdir REPLY -j RETURN
-A HI-vnet1 -p tcp -m tcp --sport 80 -m conntrack --ctstate ESTABLISHED -m
conntrack --ctdir REPLY -j RETURN
-A HI-vnet1 -j REJECT --reject-with icmp-port-unreachable
-A libvirt-host-in -m physdev --physdev-in vnet1 -g HI-vnet1
-A libvirt-in -m physdev --physdev-in vnet1 -g FI-vnet1
-A libvirt-in-post -m physdev --physdev-in vnet1 -j ACCEPT
-A libvirt-out -m physdev --physdev-out vnet1 --physdev-is-bridged -g FO-vnet1

I tried changing 'accept' into 'return' for the incoming
RELATED,ESTABLISHED rules, but that didn't help.

I also tried searching for example of more complex network filters, but all I
keep finding are the default filters of Libvirt.

Does anybody know what I'm doing wrong here? Or are there any examples of
working filters out there?

Thank you!

Wido

[0]: http://cloudstack.apache.org/