netfilter buglog - May 2013 - [Bug 820] New: Quotas not limiting the exact specified limit

https://bugzilla.netfilter.org/show_bug.cgi?id=820

           Summary: Quotas not limiting the exact specified limit
           Product: netfilter/iptables
           Version: linux-2.6.x
          Platform: x86_64
        OS/Version: Debian GNU/Linux
            Status: NEW
          Severity: critical
          Priority: P5
         Component: unknown
        AssignedTo: netfilter-buglog at lists.netfilter.org
        ReportedBy: fandaremail at gmail.com
   Estimated Hours: 0.0


Hello,
I have a problem with the quota in iptables. I have rules like bellow
for every IP, where quota is the specified limit in bytes. The problem
is that it doesnt stop when the exactly limit is reached..for example
when I set it to 1MB (quota=1048576)..it blocks the IPs when it
reaches from 1.02 to 1.04MB instead of exact 1MB..when I set the limit
to 10MB (quota=1073741824) Then it blocks the ip when it reaches 10.3
- 10.9 MB..is there a way to limit it on the exact specified amount of
data?

 /sbin/iptables -t nat -A POSTROUTING -o eth0 -s 192.168.0.2 -j MASQUERADE
 /sbin/iptables -N table1
 /sbin/iptables -A FORWARD -j table1 -d 192.168.0.2
 /sbin/iptables -A FORWARD -j table1 -s 192.168.0.2
 /sbin/iptables -A table1 -m quota --quota $quota -j ACCEPT
 /sbin/iptables -A table1 -j REJECT

I am using iptables v1.4.8 and kernel  2.6.32-5-amd64

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=820

Pablo Neira Ayuso <pablo at netfilter.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
                 CC|                            |pablo at netfilter.org

--- Comment #1 from Pablo Neira Ayuso <pablo at netfilter.org> 2013-05-15
16:35:26 CEST ---
I don't know what you're using to obtain those numbers.

Note that quota does not account the layer 2 header as iptables operates in
layer 3.

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=820

--- Comment #2 from Frantisek Remias <fandaremail at gmail.com> 2013-05-23
23:26:48 CEST ---
Here is another thing I have just noticed about this issue. The quotas are
sometimes increasing or remains the same why the byte counter in iptables are
increased? Its something what I really dont understand. Please check below

Thu May 23 23:23:12 CEST 2013
Chain 999111 (2 references)
 pkts bytes target     prot opt in     out     source               destination
 1410  686K ACCEPT     all  --  any    any     anywhere             anywhere   
        quota: 1583548 bytes
    0     0 REJECT     all  --  any    any     anywhere             anywhere   
        reject-with icmp-port-unreachable

Thu May 23 23:23:15 CEST 2013
Chain 999111 (2 references)
 pkts bytes target     prot opt in     out     source               destination
 1465  693K ACCEPT     all  --  any    any     anywhere             anywhere   
        quota: 1605551 bytes
    0     0 REJECT     all  --  any    any     anywhere             anywhere   
        reject-with icmp-port-unreachable

Thu May 23 23:23:20 CEST 2013    
Chain 999111 (2 references)
 pkts bytes target     prot opt in     out     source               destination
 1499  699K ACCEPT     all  --  any    any     anywhere             anywhere   
        quota: 1605551 bytes
    0     0 REJECT     all  --  any    any     anywhere             anywhere   
        reject-with icmp-port-unreachable

as you can see above. At 23:23:12 the bytes counter for the rule shows 686K and
quota was 1583548 bytes then 3 seconds later the bytes usage increased to 693K
and quota increased to 1605551 instead of it should decrease..how is that
possible? And then bytes increased again to 699K while quota remains the same.

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=820

Phil Oester <netfilter at linuxace.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |netfilter at linuxace.com

--- Comment #3 from Phil Oester <netfilter at linuxace.com> 2013-06-20
03:19:59 CEST ---
Quota limits should not change over time - once you set a rule to a given
quota, it should remain constant.  

Please try "watch iptables -xnvL" and see whether your quota values
are
changing.

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=820

--- Comment #4 from Phil Oester <netfilter at linuxace.com> 2013-06-28
05:23:06 CEST ---
Frantisek - any further updates on this issue?

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=820

--- Comment #5 from Frantisek Remias <fandaremail at gmail.com> 2013-06-28
21:46:01 CEST ---
Hi,

thank you for your responses on this. Thats weird becaouse the quotas are
really changing... I think that they are counting down and then when the quota
is 0 the rule disappear (which for me is correct). 
But we have to switch to a server with more CPUs/cores and now the quotas
doesnt work at all, but in the man for iptables quota is written that it doesnt
work on multi core cpus.

So I think that we cannot use iptables and quotas anymore to limit the monthly
bandwidth of IPs on our server and we have to find another soultion howto do
this. :(

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=820

--- Comment #6 from Phil Oester <netfilter at linuxace.com> 2013-06-28
22:42:56 CEST ---
Sorry I misspoke about the quotas not changing.  The issue you are encountering
is discussed in this thread:

http://marc.info/?l=netfilter-devel&m=119876883919382&w=2

There is a workaround mentioned (using taskset to force iptables to always run
on the same CPU).  

Unfortunately there is no good solution here due to the per-cpu nature of the
iptables ruleset.

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=820

--- Comment #7 from Phil Oester <netfilter at linuxace.com> 2013-07-01
22:19:54 CEST ---
I still think you are misinterpreting how the quota match works here.  If you
add this rule:  

 /sbin/iptables -A table1 -m quota --quota X -j ACCEPT

Then that rule WILL NOT CHANGE OVER TIME.  X will always remain X, and the rule
will not "disappear".  This is important, since if you want to use
iptables-save to save your ruleset, you should not have rules randomly
disappearing or changing their quota values.

What should happen, however, is that once a quota has been reached on a given
rule, it will NO LONGER MATCH.  At this point, the counters will stop
increasing for the rule.  

Are you not seeing this behavior?

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=820

Phil Oester <netfilter at linuxace.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |WONTFIX
           Severity|critical                    |normal

--- Comment #8 from Phil Oester <netfilter at linuxace.com> 2013-07-01
22:37:38 CEST ---
I see the difference - you are using an old kernel, which does not have this
patch:

   commit 49daf6a22622d4e1619aeaad5f9f0472bf89daff
   Author: Changli Gao <xiaosuo at gmail.com>
   Date:   Fri Jul 23 14:07:47 2010 +0200

     xt_quota: report initial quota value instead of current value to userspace

     We should copy the initial value to userspace for iptables-save and
     to allow removal of specific quota rules.

So on newer kernels, you will always see the same quota value everytime you run
iptables -L.  Perhaps you should upgrade?

Regardless, as previously stated, the per-cpu nature of iptables rulesets mean
you should always use the same CPU (via taskset) if you wish to see the packet
counters increasing as they should.

Closing this bug - nothing can be done unfortunately.

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.