netfilter buglog - Jan 2020 - [Bug 1400] New: "COMMIT expected at line ..." when iptables-restore 1.8.4 (nft) parses stdin with empty lines

https://bugzilla.netfilter.org/show_bug.cgi?id=1400

            Bug ID: 1400
           Summary: "COMMIT expected at line ..." when
iptables-restore
                    1.8.4 (nft) parses stdin with empty lines
           Product: iptables
           Version: unspecified
          Hardware: x86_64
               URL: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=9495
                    18
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P5
         Component: iptables-restore
          Assignee: netfilter-buglog at lists.netfilter.org
          Reporter: jamie at strandboge.com
                CC: arturo at netfilter.org

In Debian, a user reported that ufw (a frontend to iptables) was not working:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=949518

After investigating, this is a new issue with iptables-nft-restore in 1.8.4
(with
https://git.netfilter.org/iptables/commit/?id=a103fbfadf4c17b8b12caa57eef72deaaa71a18c
to fix https://bugzilla.netfilter.org/show_bug.cgi?id=1394 applied) when
parsing policy files on stdin which contain empty lines. 


Create some simple policy:

$ cat /tmp/pol
*filter
# comment
-A INPUT -j ACCEPT

COMMIT
$

With 1.8.2-4 on Debian buster, processing the file directly and on stdin are
both fine with iptables-legacy-restore and iptables-nft-restore:

$ sudo iptables-legacy-restore /tmp/pol && echo yes
yes
$ sudo iptables-nft-restore /tmp/pol && echo yes
yes

$ cat /tmp/pol | sudo iptables-legacy-restore -n && echo yes
yes
$ cat /tmp/pol | sudo iptables-nft-restore -n && echo yes
yes


With 1.8.4-2 (it has the fix for bug#1394) on sid, when processing the file
directly, it is fine:

$ sudo iptables-legacy-restore /tmp/pol && echo yes
yes
$ sudo iptables-nft-restore /tmp/pol && echo yes
yes


But processing on stdin fails with iptables-nft-restore:

$ cat /tmp/pol | sudo iptables-legacy-restore -n && echo yes
yes
$ cat /tmp/pol | sudo iptables-nft-restore -n && echo yes
iptables-nft-restore: COMMIT expected at line 4

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200121/fc2b047c/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1400

--- Comment #1 from jamie at strandboge.com ---
In looking to find a workaround for ufw to workaround this bug, I found that in
addition to blank lines in the middle of the policy causing
iptables-nft-restore to cause an error (the original report), a blank line
outside of the policy causes iptables-nft-restore to silently ignore the policy
but return a successful error code. Eg:

$ cat /tmp/pol
# this next blank line causes the file to not load

*filter
# comment
-A INPUT -j ACCEPT
COMMIT
$

Calling iptables-nft-restore on the policy file itself works fine:

$ sudo iptables-nft-restore /tmp/pol && echo yes
yes
$ sudo iptables-nft -L INPUT -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           


$ sudo iptables-nft -D INPUT -j ACCEPT  # remove the test rule
$ sudo iptables-nft -L INPUT -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination


But reading the file on stdin results in a successful return code but no rule
added:

$ cat /tmp/pol | sudo iptables-nft-restore -n && echo yes
yes
$ sudo iptables-nft -L INPUT -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination


This is a regression over 1.8.2 where it works correctly:

$ cat /tmp/pol
# this next blank line causes the file to not load

*filter
# comment
-A INPUT -j ACCEPT
COMMIT
$

$ sudo iptables-nft-restore /tmp/pol && echo yes
yes
$ sudo iptables-nft -L INPUT -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           

$ sudo iptables-nft -D INPUT -j ACCEPT  # remove the test rule
$ sudo iptables-nft -L INPUT -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

$ cat /tmp/pol | sudo iptables-nft-restore -n && echo yes
yes
$ sudo iptables-nft -L INPUT -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200122/47b9ea00/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1400

jamie at strandboge.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|"COMMIT expected at line    |regression when
                   |..." when iptables-restore  |iptables-restore 1.8.4
                   |1.8.4 (nft) parses stdin    |(nft) parses stdin with
                   |with empty lines            |empty lines

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200122/9203a639/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1400

Phil Sutter <phil at nwl.cc> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |phil at nwl.cc
         Resolution|---                         |FIXED

--- Comment #2 from Phil Sutter <phil at nwl.cc> ---
Fixed in iptables git repo (commit 8e76391096f12 ["xtables-restore: fix for
--noflush and empty lines"]), please give it a test if feasible for you.
Otherwise expect the fix to be shipped along with v1.8.5.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200212/6936cd0c/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1400

--- Comment #3 from jamie at strandboge.com ---
https://git.netfilter.org/iptables/commit/?id=8e76391096f12212985c401ee83a67990aa27a29
works great.

I patched 1.8.4-2 in Debian with it and then ran ufw's root tests (blackbox
functional tests) and they passed. Downgrading to 1.8.4-2 and the root tests
fail immediately.

Thanks!

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200213/23f23790/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1400

--- Comment #4 from Phil Sutter <phil at nwl.cc> ---
(In reply to jamie from comment #3)
> https://git.netfilter.org/iptables/commit/
> ?id=8e76391096f12212985c401ee83a67990aa27a29 works great.
> 
> I patched 1.8.4-2 in Debian with it and then ran ufw's root tests
(blackbox
> functional tests) and they passed. Downgrading to 1.8.4-2 and the root
tests
> fail immediately.
Great, thanks for testing!

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200213/fbe37197/attachment.html>