netfilter buglog - Oct 2013 - [Bug 866] New: Missing newline with --list-rules

https://bugzilla.netfilter.org/show_bug.cgi?id=866

           Summary: Missing newline with --list-rules
           Product: iptables
           Version: 1.4.x
          Platform: arm
        OS/Version: other
            Status: NEW
          Severity: normal
          Priority: P4
         Component: iptables
        AssignedTo: netfilter-buglog at lists.netfilter.org
        ReportedBy: eric.bock.1980 at gmail.com
   Estimated Hours: 2.0


It appears that Android has again been too lazy to include something that
should always exist; for example, iptables always emits this error which is
hardcoded into libc:

FIX ME! implement getprotobynumber() bionic/libc/bionic/stubs.c:456

My current filter table contains two references to quota2, and they are in the
INPUT chain.  I have no idea what inserts them; I hope Android does not use
them for something important.  The second may have originally appeared in the
OUTPUT chain and somehow migrated.

Chain INPUT (policy ACCEPT 273K packets, 85M bytes)
 pkts bytes target     prot opt in     out     source               destination
35912   47M            all  --  !lo+   any     anywhere             anywhere   
        UNKNOWN match `quota2'
 139K  164M            all  --  !lo+   any     anywhere             anywhere   
        UNKNOWN match `quota2'

This causes iptables to exit abruptly in the first line of the filter table. 
It would be better to display the entire table with an error after each line
that can't be completely interpreted; it is not guaranteed that the table is
the output of iptables commands.

[ 0 ] app_42 at android:/# iptables -vS 2>&1
-P INPUT ACCEPT -c 269887 84352741
-P FORWARD DROP -c 0 0
-P OUTPUT ACCEPT -c 152 8583
-N avastwall
-N avastwall-3g
-N avastwall-reject
-N avastwall-wifi
-N costly_shared
-N penalty_box
Can't find library for match `quota2'
-A INPUT ! -i lo+ [ 1 ] app_42 at android:/#

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=866

Phil Oester <netfilter at linuxace.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |netfilter at linuxace.com
         Resolution|                            |INVALID
    Remaining Hours|2.0                         |0.0

--- Comment #1 from Phil Oester <netfilter at linuxace.com> 2013-10-21
22:42:33 CEST ---
Android must be loading the "quota2" module from some location which
the
iptables binary does not look in.  This is not an iptables issue, as
"quota2"
is not part of the iptables distribution, and thus not supported.

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=866

Eric Bock <eric.bock.1980 at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|INVALID                     |

--- Comment #2 from Eric Bock <eric.bock.1980 at gmail.com> 2013-10-22
01:00:53 CEST ---
quota2 isn't really important to me - I just want iptables to list all the
rules instead of exiting because it can't understand the first one.

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=866

--- Comment #3 from Phil Oester <netfilter at linuxace.com> 2013-10-25
19:35:07 CEST ---
You seem to be missing the point that using iptables -S (or iptables-save) is
intended to be used to restore the rules (perhaps via iptables-restore).  If it
finds a match it does not recognize (like quota2), then it can't print out
the
options used with that match (like the quota for instance).  This is a critical
failure, and exiting is appropriate.  If you'd like to see the full ruleset,
use iptables -nvL instead, which will show you everything, even unknown matches
(which is ok, since you can't easily attempt to use this output to restore
rules).  

# iptables -nvL foo
Chain foo (0 references)
 pkts bytes target     prot opt in     out     source               destination 
    0     0            all  --  *      *       0.0.0.0/0            0.0.0.0/0  
        UNKNOWN match `quota' 
    0     0            all  --  *      *       0.0.0.0/0            0.0.0.0/0  
        UNKNOWN match `quota' 
    0     0            all  --  *      *       0.0.0.0/0            0.0.0.0/0  
        UNKNOWN match `quota' 
    0     0            all  --  *      *       1.2.3.4              0.0.0.0/0   
    0     0            all  --  *      *       1.2.3.4              0.0.0.0/0   
    0     0            all  --  *      *       1.2.3.4              0.0.0.0/0   
    0     0            all  --  *      *       1.2.3.4              0.0.0.0/0

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=866

--- Comment #4 from Eric Bock <eric.bock.1980 at gmail.com> 2013-10-25
21:45:50 CEST ---
I don't have much control over whether Android puts garbage in the firewall
rules.  Currently, in order to save the complete tables I am forced to delete
the quota2 rules, which Android probably inserts at boot time.  Since Android
is more prone to sudden rebooting than early Windows, the quota2 rules tend to
exist.

Regarding the exit from iptables: when it encounters the quota2 rule, iptables
emits a warning on stderr and an unterminated line on stdout.  The last line
easily becomes overwritten by the shell prompt and causes bash to become
confused about the cursor location:

[ 0 ] app_42 at android:/# iptables -S OUTPUT
-P OUTPUT ACCEPT
Can't find library for match `quota2'
-A OUTPUT ! -o lo+ [ 1 ] app_42 at android:/# iptables -S OUTPUT
-P OUTPUT ACCEPT
Can't find library for match `quota2'
[ 1 ] app_42 at android:/#
          ] || echo $x; done

(following some history navigation)

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=866

--- Comment #5 from Eric Bock <eric.bock.1980 at gmail.com> 2013-10-26
03:01:21 CEST ---
If the intent is to ensure the output of `iptables -S` is valid input for
restoring the current firewall state, partial output is not helpful.  For
example, a boot script that (apparently, naively) expects `iptables -S` to
always return successfully after outputting the entire filter table might
backup the following incorrect rules in place of previously correct rules:

-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-A INPUT ! -i lo+

["Can't find library for match `quota2'" sent to the terminal,
if there is one]

A supposed recovery from backup proceeds like this:

[ 0 ] app_42 at android:/# iptables -F INPUT
[ 0 ] app_42 at android:/# iptables -vL INPUT
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

[...]

[ 0 ] app_42 at android:/# iptables -A INPUT ! -i lo+
[ 0 ] app_42 at android:/# iptables -vL INPUT
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0            all  --  !lo+   any     anywhere             anywhere
[ 0 ] app_42 at android:/#

This is not the same rule that existed previously; if quota2 becomes available
it will have no effect.  In the meantime the firewall is permitting all traffic
because the stored list was essentially empty.

There is probably some pathological rule which could be benign while present in
the filter table but which becomes harmful if used as truncated by `iptables
-S`.  iptables cannot know when the effect of a partial rule might be more
desirable than its complete inclusion or omission.

It would be better to output every rule completely without regard for whether
iptables would currently accept them on the command line.  They might all be
valid at restore time.  Complete output could be processed by some other
command (such as `grep -v quota2`) to produce the rules desired, and would also
help for simple inspection of errors introduced into the table.

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

https://bugzilla.netfilter.org/show_bug.cgi?id=866

Pablo Neira Ayuso <pablo at netfilter.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
                 CC|                            |pablo at netfilter.org
         Resolution|                            |WONTFIX

--- Comment #6 from Pablo Neira Ayuso <pablo at netfilter.org> 2013-10-26
12:01:04 CEST ---
(In reply to comment #5)
[...]
> It would be better to output every rule completely without regard for
whether
> iptables would currently accept them on the command line.  They might all
be
> valid at restore time.  Complete output could be processed by some other
> command (such as `grep -v quota2`) to produce the rules desired, and would
also
> help for simple inspection of errors introduced into the table.
iptables *cannot* output the rule including the quota2 match if it doesn't
locate the quota2 extension in its typical library location (commonly,
/usr/lib/xtables/libxt_*.so) or, alternatively, if that support is not compiled
built-in. The extension allows iptables to interpret the rule in binary format
and translate it to human readable syntax, if the "interpreter" (the
extension
in this case) is not found, then you hit problems like this.

I guess that Android is doing some nasty hack, probably adding the rule using
the quota2 match via the internal library libiptc (which was not ever intended
to be a public library) but then fails to find the extension to interpret it
due to broken packaging.

Please, file a bug to your Android vendor, they seem to be providing a *pretty
broken* installation of iptables, including that custom quota2 extension that
we don't support in mainstream Linux kernels and iptables, so they can fix
it.

-- 
Configure bugmail: https://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.