netfilter buglog - Nov 2019 - [Bug 1382] New: nftables.py cmd leaking memory when ruleset contain mapping ip length to range with high limit 65535

https://bugzilla.netfilter.org/show_bug.cgi?id=1382

            Bug ID: 1382
           Summary: nftables.py cmd leaking memory when ruleset contain
                    mapping ip length to range with high limit 65535
           Product: nftables
           Version: unspecified
          Hardware: x86_64
                OS: Gentoo
            Status: NEW
          Severity: major
          Priority: P5
         Component: nft
          Assignee: pablo at netfilter.org
          Reporter: karel at unitednetworks.cz

Created attachment 574
  --> https://bugzilla.netfilter.org/attachment.cgi?id=574&action=edit
example of nftables.py leaking memory

System:

Gentoo 5.3.10 x86_64
nft up to date from GIT as of 12.11.2019


Overwiew:

nft commands which change ruleset leak memory when running through nftables.py
if ruleset contain rule like this:

add rule ip table1 chain1 counter name ip length map { 0-65535:
"counter1" }

But no leaking happens when rule is changed to:

add rule ip table1 chain1 counter name ip length map { 0-65534:
"counter1" }


Detailed info:

Consider this ruleset:

add table ip table1
add set ip table1 set1 { type ipv4_addr; }
add chain ip table1 chain1 { type filter hook forward priority 0; }
add counter ip table1 counter1
add rule ip table1 chain1 counter name ip length map { 0-65535:
"counter1" }

Then several iterations of adding and deleting set items like this:

add element table1 set1 { 1.1.1.1 }
delete element table1 set1 { 1.1.1.1 }

will show process is leaking memory.


See attached example.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20191112/45ec5916/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1382

Karel Rericha <karel at unitednetworks.cz> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|nftables.py cmd leaking     |nftables.py cmd leaking
                   |memory when ruleset contain |memory when ruleset
                   |mapping ip length to range  |contains mapping ip length
                   |with high limit 65535       |to range with high limit
                   |                            |65535

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20191112/9183a0ee/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1382

--- Comment #1 from Karel Rericha <karel at unitednetworks.cz> ---
Forget to add:

- Python version of tested system is CPython 3.6.9

- we have observed this memory leak on several other servers with kernels
4.19.x and older version of nft utility (about year old)

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20191113/4026f926/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1382

--- Comment #2 from Karel Rericha <karel at unitednetworks.cz> ---
For kernel 5.7.10 and latest nft (3.8.2020) iteration leak is somewhat smaller,
but still present. Looks like recent commits fixed some memory leaks.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200803/712cc4e0/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1382

Pablo Neira Ayuso <pablo at netfilter.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED

--- Comment #3 from Pablo Neira Ayuso <pablo at netfilter.org> ---
Testing:

add table ip table1
add set ip table1 set1 { type ipv4_addr; }
add chain ip table1 chain1 { type filter hook forward priority 0; }
add counter ip table1 counter1
add rule ip table1 chain1 counter name ip length map { 0-65535:
"counter1" }

Then:

nft list ruleset

shows no memleaks after this patch:

https://patchwork.ozlabs.org/project/netfilter-devel/patch/20200805093312.26212-1-pablo
at netfilter.org/

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200805/f812d139/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1382

--- Comment #4 from Karel Rericha <karel at unitednetworks.cz> ---
I can confirm that after applying Pablo's patch there are no memory leaks
with
given example:

Before patch:

Iterations: 10000
Total time:     0.5286474227905273
Iteration time: 5.286474227905273e-05
Total leak:     4874240
Iteration leak: 492.34747474747473

After patch:

Iterations: 10000
Total time:     0.5286917686462402
Iteration time: 5.2869176864624025e-05
Total leak:     0
Iteration leak: 0.0

Thanks a lot Pablo!

I will close this after patch will land in nftables git.

BTW Pablo will you be so kind and look at this bug too:

https://bugzilla.netfilter.org/show_bug.cgi?id=1386

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200806/fcf58dc2/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1382

Karel Rericha <karel at unitednetworks.cz> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200806/cb49c425/attachment.html>