netfilter buglog - Mar 2017 - [Bug 1128] New: ip6_tables connmark or connlabel never matches

https://bugzilla.netfilter.org/show_bug.cgi?id=1128

            Bug ID: 1128
           Summary: ip6_tables connmark or connlabel never matches
           Product: netfilter/iptables
           Version: unspecified
          Hardware: x86_64
                OS: SuSE Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: ip6_tables (kernel)
          Assignee: netfilter-buglog at lists.netfilter.org
          Reporter: jimc at jfcarter.net

Created attachment 497
  --> https://bugzilla.netfilter.org/attachment.cgi?id=497&action=edit
Test ruleset using connmark, should fail reliably

This is for kernel 4.10.1-1-default from OpenSuSE "Tumbleweed" in a 
background of OpenSuSE "Leap" 42.1.  The rules were loaded by 
ip6tables-restore from iptables-1.6.1-1.1 and matching versions for
libxtables12, xtables-plugins, libnetfilter_conntrack3-1.0.6-1.3 .
Same symptoms using iptables-1.4.21-4.1 and friends.  
It is running on a KVM/qemu virtual machine (Intel x86_64); the same
problem is seen on bare metal with the same kernel.  4.8.13-1-default
is the last symptom-free kernel; the problem began after I updated to
4.9.9-default and continued in 4.10.1-1-default.  (I'm using a more
recent kernel than the distro to get support for a wireless NIC and a
touchpad.)  

My firewall sets connmarks, which are not matched when they should be.
Hoping for a quick fix I revised the code to use connlabels, which also
are not matched.  I wrote two files of test rules which set a connmark
or connlabel, then accept the packet if it has the mark or label, and
drop it otherwise.  All packets (ipv6-icmp echo request, neighbor
advertisement, and a variety of TCP and UDP) are dropped.  According to
"conntrack -f ipv6 -o labels -L", the mark is set on the conntrack
correctly according to which set of rules is loaded, or the expected
labels are set.  All of these fail to execute their -j targets: 
--mark 0x100/0x100 (test if the mark bit is present);
! --mark 0x000/0x100 (test if the mark is not absent); 
--mark 0x000/0x100 (test if the mark is absent, if the mark were not
getting set), and similarly for the connlabel.  The mark or label is
reported as set per "conntrack -L".  

This is all for IPv6.  On IPv4 I use identical logic except for two or
three family-specific rules, e.g. "-p icmp" vs. "-p
ipv6-icmp" and
"--reject-with icmp6-port-unreachable".  It works perfectly: connmarks
are matched as they should be.  Proven because I can connect to the 
guest using "slogin -4".  

Are there other symptoms?  The -i match is definitely happening, proven
because connmark bits conditional on the interface (in the full firewall
rule set) are being set.  Jumping to other chains (-j or -g) is
definitely happening.  Intrinsic targets ACCEPT, DROP and LOG are
definitely happening as they should (if the rule executes them).  Suppose
matches other than connmark and connlabel were wrongly not executing
their targets.  It would be hard to be sure of that symptom when the
control flow through the code was uncertain due to failure to match the
connmarks.

Looking in /usr/src/linux-4.10.1/net/netfilter/xt_connmark.c, also
xt_connlabel.c, I find clean object-oriented code called through the
generic struct xt_match or xt_target virtual tables, with no visible
bugs, i.e. the problem is probably elsewhere.  At this point the obvious
strategy is GIT bifurcation, but I've never done that before, and so I
figured it would be wiser to ask someone who might recognize the issue.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20170310/c68b1efd/attachment-0001.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1128

--- Comment #1 from Jim Carter <jimc at jfcarter.net> ---
Created attachment 498
  --> https://bugzilla.netfilter.org/attachment.cgi?id=498&action=edit
Test ruleset using connlabel

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20170310/f05b1717/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1128

Florian Westphal <fw at strlen.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|netfilter-buglog at lists.netf |fw at strlen.de
                   |ilter.org                   |
                 CC|                            |fw at strlen.de

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20170310/0edf1fcb/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1128

--- Comment #2 from Florian Westphal <fw at strlen.de> ---
(In reply to Jim Carter from comment #1)
> Created attachment 498 [details]
> Test ruleset using connlabel
Thanks, the reproducer works for me.
This is caused by

netfilter: connmark: ignore skbs with magic untracked conntrack objects

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=fb9c9649a1d0a65a8f94f784aa18252a0dd584c1

but unfortunately that change is/was intentional.

The problem is that in ipv6, mac address <-> ipv6 address resolution is
handled
by icmpv6, and netfilters connection tracker intentionally ignores them
(ipv4 uses arp which isn't handled by iptables).

Before the commit, this worked because the mark would be added to the percpu
untracked object.  It never worked for connlabels as the match always contained
the ct_is_untracked() test.

The test works fine when adding

ip6tables -I INPUT -p icmpv6 -m conntrack --ctstate UNTRACKED -j ACCEPT

to pass neigh resolution packets.

I see no way to fix this, reverting is possible of course but has other
side effects, the percpu object is never reset so when doing something like

-m <some condition> -j CONNMARK --set-mark 42
-m connmark --mark 42

... the 2nd rule will match other UNTRACKED packets too.
Resetting the untracked object is possible, but will need more code to avoid
problems when NFQUEUE gets used (so multiple packets can refer to same
untracked
entry).

Furthermore, this would force us to retain the UNTRACKED object, so we won't
be
able to remove it as done in this pending patch (not yet applied):

https://patchwork.ozlabs.org/patch/736578/

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20170310/4b2fb90c/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1128

--- Comment #3 from Jim Carter <jimc at jfcarter.net> ---
Thank you for the quick and clear explanation.  The workaround works in
the complete firewall ruleset (if you put it in the right place).  

I saw your blog post on nftables (2016-10-28)
https://developers.redhat.com/blog/2016/10/28/what-comes-after-iptables-its-successor-of-course-nftables/
I look forward to using the new framework; it looks a lot cleaner than
what we have now.  But my firewall uses both the IPSec policy match and
TCPMSS clamping for tunnels, so adoption can't be immediate.  

Thanks to you guys for your work on netfilter.  It's a key component of
defense in depth, which in today's political climate is particularly 
necessary.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20170312/10199fdf/attachment.html>