netfilter buglog - Feb 2020 - [Bug 1404] New: Problems with dynamically managing interval sets with auto-merge

https://bugzilla.netfilter.org/show_bug.cgi?id=1404

            Bug ID: 1404
           Summary: Problems with dynamically managing interval sets with
                    auto-merge
           Product: nftables
           Version: unspecified
          Hardware: All
                OS: Debian GNU/Linux
            Status: NEW
          Severity: blocker
          Priority: P5
         Component: nft
          Assignee: pablo at netfilter.org
          Reporter: pc at hillside.co.uk

When element contents are auto-merged, a flush action will remove set elements
but seems to not permit elements to be added in the same command sequence.

Step 1:
Place following into a file (assumes ip filter table exists) to create a set
with a merged interval.

add set ip filter w_all {type ipv4_addr; flags interval;auto-merge}
add element ip filter w_all {10.10.10.10,10.10.10.11}

Step 2:
Assume time passes and a change is needed. Now in a second file, execute:

flush set ip filter w_all
add element ip filter w_all {10.10.10.10,10.10.10.253}

The add element command fails with

n2:2:30-40: Error: interval overlaps with an existing one
add element ip filter w_all {10.10.10.10,10.10.10.253}
                             ^^^^^^^^^^^
n2:2:1-55: Error: Could not process rule: File exists
add element ip filter w_all {10.10.10.10,10.10.10.253}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
and the set is unchanged.

Adding 

flush set ip filter w_all

to the end of the first command set does result in an empty set. So it's
flushing, but I am guessing there is some state which retains the auto-merge
information which isn't cleared and this causes the second add element to
fail.

Workaround 1:
Removing auto-merge from the set definition allows the contents to be replaced.

Workaround 2:
A workaround seems to be to split the second command set into two separate
calls to nft. First to flush the set. Then a second action to re-populate.
However, I really expect the flush to work.

Comments: An attempt to delete one element of an auto-merged element fails. I
think that this is reasonable. 

# nft -v
nftables v0.9.3 (Topsy)

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200203/9cbb8737/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1404

kfm at plushkava.net changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kfm at plushkava.net

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200205/28a70a5d/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1404

Frank Myhr <fmyhr at fhmtech.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fmyhr at fhmtech.com

--- Comment #1 from Frank Myhr <fmyhr at fhmtech.com> ---
Hit this one today. I've nothing to add to the thorough bug report, which
describes the behavior I observed to a tee.

Comments on workarounds:

1: Can confirm that removing auto-merge fixed the problem in my case as well.

2: Flushing the set in a separate call to nft prior to re-populating it loses
atomicity of the combined operation. If the set is used as an ACL (likely, as
this is a firewall after all), the brief time interval during which the set is
empty can result in unintended network access (or unintended blocking of such
access).

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200507/fa592210/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1404

Pablo Neira Ayuso <pablo at netfilter.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED

--- Comment #2 from Pablo Neira Ayuso <pablo at netfilter.org> ---
Please, check if your problem is solved in current upstream nftables git
snapshot, this fix is targetting at solving the problem you describe.

http://git.netfilter.org/nftables/commit/?id=40ef308e19b6db02017a8a650406b0c6d37be750

Thanks for reporting.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200721/2fbf7bc9/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1404

Timo Sigurdsson <public_timo.s at silentcreek.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |public_timo.s at silentcreek.d
                   |                            |e

--- Comment #3 from Timo Sigurdsson <public_timo.s at silentcreek.de> ---
(In reply to Pablo Neira Ayuso from comment #2)
> Please, check if your problem is solved in current upstream nftables git
> snapshot, this fix is targetting at solving the problem you describe.
> 
> http://git.netfilter.org/nftables/commit/
> ?id=40ef308e19b6db02017a8a650406b0c6d37be750
> 
> Thanks for reporting.
Hi Pablo,

I tried this one, too, and sadly, it doesn't seem fixed. 

Again, for reference, the details of my build are:
Ubuntu 20.04 with nftables and libnftnl built from source (at commits 7c9bef0
and 330ca1c respectively) and `nft -v' showing: nftables v0.9.6 (Capital
Idea
#2).

Regards,

Timo

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200730/7adc9da2/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1404

--- Comment #4 from Pablo Neira Ayuso <pablo at netfilter.org> ---
Already tested as described at:

https://bugzilla.netfilter.org/show_bug.cgi?id=1431

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200730/6033ae3d/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1404

Pablo Neira Ayuso <pablo at netfilter.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |DUPLICATE

--- Comment #5 from Pablo Neira Ayuso <pablo at netfilter.org> ---


*** This bug has been marked as a duplicate of bug 1431 ***

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200730/35d120b1/attachment-0001.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1404

kfm at plushkava.net changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://bugzilla.netfilter.
                   |                            |org/show_bug.cgi?id=1392

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200730/365500e5/attachment.html>