netfilter buglog - Nov 2017 - [Bug 1201] New: Some filters randomly do not work since version 0.8

https://bugzilla.netfilter.org/show_bug.cgi?id=1201

            Bug ID: 1201
           Summary: Some filters randomly do not work since version 0.8
           Product: nftables
           Version: unspecified
          Hardware: x86_64
                OS: Gentoo
            Status: NEW
          Severity: major
          Priority: P5
         Component: nft
          Assignee: pablo at netfilter.org
          Reporter: sautier.louis at gmail.com

Hello,
Since I upgraded to version 0.8, I have been experiencing weird behaviour with
some filters not matching. I think the issue is only present with filters for
tcp ports but this is just a guess.
Here is what my ip input filter chain looks like:
table ip filter {
        chain INPUT {
                type filter hook input priority 0; policy accept;
                ct state established,related accept
                iifname "eth0" tcp dport { 22, 80, 443 } counter
accept
                iifname "lo" accept
                tcp dport 80 counter
                iifname "eth0" tcp dport 80 counter
                iifname "eth0" tcp dport { 80, 111 } counter
                iifname "eth0" tcp dport { 80, 111, 112, 113, 114,
115, 117 }
counter
                reject with tcp reset
                reject
        }
}
The rules with counters at the end are here for debugging purposes, they
shouldn't match a lot of traffic since ports 11* are unused and the third
rule
should accept packets sent to port 80.

However, it seems that some rules tend not to match:
                iifname "eth0" tcp dport { 22, 80, 443 } counter
packets 0
bytes 0 accept
                tcp dport 80 counter packets 264 bytes 15756
                iifname "eth0" tcp dport 80 counter packets 264 bytes
15756
                iifname "eth0" tcp dport { 80, 111 } counter packets 0
bytes 0
                iifname "eth0" tcp dport { 80, 111, 112, 113, 114,
115, 117 }
counter packets 0 bytes 0


If I reload the rules a few times, I'll sometimes see the expected
behaviour:
                iifname "eth0" tcp dport { 22, 80, 443 } counter
packets 31
bytes 1852 accept
                tcp dport 80 counter packets 0 bytes 0
                iifname "eth0" tcp dport 80 counter packets 0 bytes 0
                iifname "eth0" tcp dport { 80, 111 } counter packets 0
bytes 0
                iifname "eth0" tcp dport { 80, 111, 112, 113, 114,
115, 117 }
counter packets 0 bytes 0

Sometimes, the accept rule won't match but the { 80, 111 } one will:
                iifname "eth0" tcp dport { 22, 80, 443 } counter
packets 0
bytes 0 accept
                tcp dport 80 counter packets 4 bytes 240
                iifname "eth0" tcp dport 80 counter packets 4 bytes
240
                iifname "eth0" tcp dport { 80, 111 } counter packets 4
bytes
240
                iifname "eth0" tcp dport { 80, 111, 112, 113, 114,
115, 117 }
counter packets 0 bytes 0


I am running Gentoo with kernel 4.13.12, I also had the issue with 4.13.11. I
could include my kernel config but I'm pretty sure the problem is with
nftables
itself.
My nftables 0.8 is compiled with:
./configure --prefix=/usr --build=x86_64-pc-linux-gnu
--host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info
--datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib
--disable-dependency-tracking --disable-silent-rules
--docdir=/usr/share/doc/nftables-0.8-r3
--htmldir=/usr/share/doc/nftables-0.8-r3/html --libdir=/usr/lib64
--sbindir=/sbin --disable-pdf-doc --disable-debug --with-cli --without-mini_gmp

I am unable to reproduce the problem with version 0.7 compiled with these (the
same parameters as 0.8 except for docdir and htmldir):
./configure --prefix=/usr --build=x86_64-pc-linux-gnu
--host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info
--datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib
--disable-dependency-tracking --disable-silent-rules
--docdir=/usr/share/doc/nftables-0.7 --htmldir=/usr/share/doc/nftables-0.7/html
--libdir=/usr/lib64 --sbindir=/sbin --disable-pdf-doc --disable-debug
--with-cli --without-mini_gmp

I may try to run a bisect on this but if someone from the project could help
me, that would save me quite a bit of time.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20171109/0f2fe2b3/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1201

--- Comment #1 from Louis Sautier <sautier.louis at gmail.com> ---
61428af7486defec6adafc9b6a2ee0602fd98b48 is the first bad commit
commit 61428af7486defec6adafc9b6a2ee0602fd98b48
Author: Pablo Neira Ayuso <pablo at netfilter.org>
Date:   Fri May 26 11:49:19 2017 +0100

    netlink: add size description for constant sets

    The kernel side can make better decisions with this information when
    selecting the right backend, so add this information to the set netlink
    message.

    Signed-off-by: Pablo Neira Ayuso <pablo at netfilter.org>

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20171109/3df121fa/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1201

--- Comment #2 from Louis Sautier <sautier.louis at gmail.com> ---
I compiled v0.8 but the offending commit and I don't have the issue any
more.
If anyone runs into the same issue, they can apply this patch instead of
downgrading:

diff --git a/src/netlink.c b/src/netlink.c
index 2882190..59e8918 100644
--- a/src/netlink.c
+++ b/src/netlink.c
@@ -1293,8 +1293,6 @@ static int netlink_add_set_batch(struct netlink_ctx *ctx,
                if (set->desc.size != 0)
                        nftnl_set_set_u32(nls, NFTNL_SET_DESC_SIZE,
                                          set->desc.size);
-       } else if (set->init) {
-               nftnl_set_set_u32(nls, NFTNL_SET_DESC_SIZE,
set->init->size);
        }

        udbuf = nftnl_udata_buf_alloc(NFT_USERDATA_MAXLEN);

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20171109/1feda06a/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1201

--- Comment #3 from Louis Sautier <sautier.louis at gmail.com> ---
Apparently this is linked to
https://git.netfilter.org/nftables/commit/?id=61428af7486defec6adafc9b6a2ee0602fd98b48
With a patched kernel, I do not have the issue with vanilla nftables 0.8.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20171109/c7bf1028/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1201

--- Comment #4 from Louis Sautier <sautier.louis at gmail.com> ---
I meant this commit:
https://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf.git/commit/?id=0414c78f14861cb704d6e6888efd53dd36e3bdde

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20171109/331e7fc8/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1201

Pablo Neira Ayuso <pablo at netfilter.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED

--- Comment #5 from Pablo Neira Ayuso <pablo at netfilter.org> ---
(In reply to Louis Sautier from comment #4)
> I meant this commit:
> https://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf.git/commit/
> ?id=0414c78f14861cb704d6e6888efd53dd36e3bdde
This fix has been included in Linux kernel release 4.13.13.

Please, confirm this kernel is working fine for you. Thanks!

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20171115/4b21730f/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1201

Pablo Neira Ayuso <pablo at netfilter.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bugzilla at hard-wired.net

--- Comment #6 from Pablo Neira Ayuso <pablo at netfilter.org> ---
*** Bug 1200 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20171115/55297bb5/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1201

Pablo Neira Ayuso <pablo at netfilter.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |makovick at gmail.com

--- Comment #7 from Pablo Neira Ayuso <pablo at netfilter.org> ---
*** Bug 1199 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20171115/c867041d/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1201

Louis Sautier <sautier.louis at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|ASSIGNED                    |RESOLVED

--- Comment #8 from Louis Sautier <sautier.louis at gmail.com> ---
I only tested 4.13.12 with the patch and just upgraded to 4.14.0 which works
fine. I'll close this.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20171115/2d082475/attachment.html>