netfilter buglog - Nov 2013 - [Bug 876] New: bizarre handling of "related" connection packets (wrong OUTPUT interface assigned)

https://bugzilla.netfilter.org/show_bug.cgi?id=876

           Summary: bizarre handling of "related" connection packets
                    (wrong OUTPUT interface assigned)
           Product: netfilter/iptables
           Version: unspecified
          Platform: x86_64
        OS/Version: Fedora
            Status: NEW
          Severity: normal
          Priority: P5
         Component: netfilter hooks
        AssignedTo: netfilter-buglog at lists.netfilter.org
        ReportedBy: mr.dash.four at googlemail.com
   Estimated Hours: 0.0


A bit of background: on one of my firewall machines, there are 7 different
interfaces - 2 outward-facing, connected to the outside world (eth0 and tun0)
and 5 inward-facing interfaces, connected to various internal networks
(eth1-4). I also have a local zone on lo (the loopback interface) - called
"fw2local" and "local2fw" with various rules in existence.
It is also worth
mentioning that I have strict control of all related connections, with the
appropriate logging set on all zones/interfaces.

Now, when say, eth3 and eth4 have established connections to various IP
addresses to the outside world via eth0 (with the appropriate masquerading set)
and the link on eth0 drops, I get a flurry of about 30-40 entries in my logs
consisting of icmp type 3, code 1 messages with source address set as the eth0
IP address and destination address set as the originating IP address, belonging
to subnets of which my eth3 and eth4 interfaces are part of. So far so good and
is more or less what I expect to see in such circumstances.

What is rather bizarre, however, is that all of this goes through lo (the
loopback)!

Yes, that's right - I am seeing these logs appear on my fw2local zone (with
state = RELATED), which is responsible for packets from/to the local zone
("-i
lo" and "-o lo"), consisting of a single interface - the
loopback, not the
zones which are responsible for handling packets from/to eth3 or eth4.

In other words, I expected to see these packets appear in one of fw2eth3 (state
= RELATED), fw2eth4 (state = RELATED), eth02eth3 (state = RELATED) or eth02eth4
(state = RELATED) zones, not on my fw2local zone.

The actual logs (see below) confirm that the out interface is indeed the
loopback (lo), even though none of the addresses involved are on the loopback
interface (source/destination IP addresses of either related or the actual
connection) and the destination address is not present on any of the other
interfaces either (I am aware that if source/destination addresses are on the
local machine, then these packets can be treated as "local",  but that
is not
the case here).

None of these packets reach their destination, which is hardly surprising since
the out interface is the loopback.

So, to summarise, the original packet flow goes like this: (internal machine
->
eth1 (fw) -> eth0 (fw) -> NAT -> external IP address)

When the connection on eth0 drops, I start getting quite a few logs like these
(produced with ulogd2 - pretty-print plugin):

timestamp=2013/10/09-00:04:57,raw.pktlen=576,raw.pktcount=1,oob.prefix=fw2local:DROP:,oob.time.sec=1381273497,oob.time.usec=72127,oob.mark=0,oob.ifindex_out=1,oob.hook=3,oob.family=2,oob.protocol=0,raw.label=0,ip.saddr=<eth0>,ip.daddr=<remote_IP_eth1_subnet>,ip.protocol=1,ip.tos=192,ip.ttl=64,ip.totlen=576,ip.ihl=5,ip.csum=31876,ip.id=53893,ip.fragoff=0,icmp.type=3,icmp.code=1,icmp.csum=30221,oob.protocol=0,o.ip.saddr=<remote_IP_eth1_subnet>,o.ip.daddr=<external_IP>,o.ip.protocol=6,o.ip.tos=0,o.ip.ttl=63,o.ip.totlen=638,o.ip.ihl=5,o.ip.csum=15844,o.ip.id=42550,o.ip.fragoff=16384,o.tcp.sport=38327,o.tcp.dport=443,o.tcp.seq=3400886979,o.tcp.ackseq=4068520201,o.tcp.window=166,o.tcp.offset=0,o.tcp.reserved=0,o.tcp.urg=0,o.tcp.ack=1,o.tcp.psh=1,o.tcp.rst=0,o.tcp.syn=0,o.tcp.fin=0,o.tcp.res1=0,o.tcp.res2=0,o.tcp.csum=26701,oob.in=,oob.out=lo,ip.saddr.str=<eth0>,ip.daddr.str=<remote_IP_eth1_subnet>,oob.protocol=0

Please note that all of the "o.*" properties are for the original
connection
for which the icmp message was issued; "<eth0>" is my
external-facing IP on
eth0, "<remote_IP_eth1_subnet>" is remote internal IP which has
the same subnet
as eth1 (i.e. it is reachable via eth1), "<external_IP>" is a
remote host.

For the record, I don't think this is a routing problem: "ip route ls
table
local" gives me what I expect to see - a group of 3 routes per interface.
For
example, if we assume eth0 to have 10.1.0.1/24, then I have something like:

broadcast 10.1.0.0 dev eth0  proto kernel  scope link  src 10.1.0.1
local 10.1.0.1 dev eth0  proto kernel  scope host  src 10.1.0.1
broadcast 10.1.0.255 dev eth0  proto kernel  scope link  src 10.1.0.1

The above repeats for each interface on that machine. The only exception to
this is lo, where I have:

broadcast 127.0.0.0 dev lo  proto kernel  scope link  src 127.0.0.1
local 127.0.0.0/8 dev lo  proto kernel  scope host  src 127.0.0.1
local 127.0.0.1 dev lo  proto kernel  scope host  src 127.0.0.1
broadcast 127.255.255.255 dev lo  proto kernel  scope link  src 127.0.0.1

"ip route ls cache" returns nothing.

The kernel version I am using is 3.11.1.

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.