netfilter buglog - Oct 2013 - [Bug 859] New: bitmap:port appears to be using 8 bytes of kernel memory per port, rather than 1 bit.

https://bugzilla.netfilter.org/show_bug.cgi?id=859

           Summary: bitmap:port appears to be using 8 bytes of kernel
                    memory per port, rather than 1 bit.
           Product: ipset
           Version: unspecified
          Platform: x86_64
        OS/Version: All
            Status: NEW
          Severity: critical
          Priority: P5
         Component: default
        AssignedTo: netfilter-buglog at lists.netfilter.org
        ReportedBy: quentin at armitage.org.uk
   Estimated Hours: 0.0


Using ipset version 6.19 on kernel version 3.6.11, a bitmap:ip range 1-65535 is
reported to be using 8304 bytes of memory. With a kernel version 3.11.3, the
same size bitmap:ip is reported to be using 524424 bytes of memory.

# uname -r
3.6.11-4.fc16.x86_64
# ipset create fred bitmap:port range 1-65535
# ipset list fred
Name: fred
Type: bitmap:port
Revision: 0
Header: range 1-65535
Size in memory: 8304
References: 0
Members:

# uname -r
3.11.3-201.fc19.x86_64
# ipset create fred bitmap:port range 1-65535
# ipset list fred
Name: fred
Type: bitmap:port
Revision: 1
Header: range 1-65535
Size in memory: 524424
References: 0
Members:

A 3.7.5 kernel reports memory utilisation of 8260, so slightly lower than
3.6.11. Unfortunately I don't have any kernels between 3.7.5 and 3.11.3
running.

Further checks show that for every additional port in the bitmap, kernel 3.11.3
reports adding 8 bytes to the size of the bitmap.

A bitmap:ip does NOT have the same problem.

I assume it is a problem in the kernel, since I am using ipset version 6.19 on
both systems (I have also tried ipset v6.20 on the 3.11.3 system, but it shows
the same problem).

Is the 3.11.3 kernel using an excessive amount of memory for bitmap:port, or is
it incorrectly reporting the amount of memory used?

I've marked this as critical in case the kernel is using the excessive
amount
of memory. If it is just the amount of memory being used, then I'm happy for
the severity to be lowered.

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=859

Jozsef Kadlecsik <kadlec at netfilter.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kadlec at netfilter.org

--- Comment #1 from Jozsef Kadlecsik <kadlec at netfilter.org> 2013-10-18
14:22:18 CEST ---
You are completely right, too large memory was allocated - the code slipped in
at the restructuring of the bitmap types creation. It is fixed now in the ipset
git tree.

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=859

Jozsef Kadlecsik <kadlec at netfilter.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=859

--- Comment #2 from Quentin Armitage <quentin at armitage.org.uk>
2013-10-19 13:48:50 CEST ---
(In reply to comment #1)
> You are completely right, too large memory was allocated - the code slipped
in
> at the restructuring of the bitmap types creation. It is fixed now in the
ipset
> git tree.
Many thanks for the prompt fix. Will this patch be pushed upstream for the 3.10
and 3.11 kernels?

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.