Shorewall users - Jun 2008 - strange bug/problem with shorewall after centos-5.2 update

hi,
we use shorewall for many years. noew we update our firewall to centos 
5.2. where run shorewall-lite and there is an internal server which is 
the administrative system. now the following happend. after the firewall 
reboot shorewall (or iptables or the system denied all external 
connections (although shorewall-lite is started). now if i restart it:
/sbin/service shorewall-lite restart
then everything work. or if from the administrative system i issue a:
/sbin/shorewall reload -s -c portal
than it also works again. so each of the above command is enough. so it 
was easy to find a workaround i simple put into rc.local:
/sbin/service shorewall-lite restart
but imho it''s still a bug and i don''t know how to find the
reason.
the only difference what i find in the sysinit script is -f option to 
shorewall but in /var/lib/shorewall-lite/ the file firewall and restore 
are the same.
if i comment out the -f option then it''s working without any workaround
(or this is the workaround).
so what can be the reason?

-- 
   Levente                               "Si vis pacem para bellum!"

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It''s the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

Farkas Levente wrote:
> hi,
> we use shorewall for many years. noew we update our firewall to centos 
> 5.2. where run shorewall-lite and there is an internal server which is 
> the administrative system. now the following happend. after the firewall 
> reboot shorewall (or iptables or the system denied all external 
> connections (although shorewall-lite is started). now if i restart it:
> /sbin/service shorewall-lite restart
> then everything work. or if from the administrative system i issue a:
> /sbin/shorewall reload -s -c portal
> than it also works again. so each of the above command is enough. so it 
> was easy to find a workaround i simple put into rc.local:
> /sbin/service shorewall-lite restart
> but imho it''s still a bug and i don''t know how to find
the reason.
Compare the output of ''shorewall-lite dump'' before and after
the restart.
> the only difference what i find in the sysinit script is -f option to 
> shorewall but in /var/lib/shorewall-lite/ the file firewall and restore 
> are the same.
The files /var/lib/shorewall-lite/firewall and /var/lib/shorewall/restore 
are supposed to be the same if you have done a ''shorewall-lite
save''.
> if i comment out the -f option then it''s working without any
workaround
> (or this is the workaround).
> so what can be the reason?
> 
The file /var/lib/shorewall-lite/.iptables-restore-input is probably wrong. 
But since I can''t see it, I can''t tell you what is wrong with
it.

You can try this experiment:

a) cd /var/lib/shorewall-lite
b) mv .iptables-restore-input bad-input
c) shorewall-lite save
d) diff -au bad-input .iptables-restore-input

What are the differences?

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key



-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It''s the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

Tom Eastep wrote:
> The file /var/lib/shorewall-lite/.iptables-restore-input is probably 
> wrong. But since I can''t see it, I can''t tell you what is
wrong with it.
> 
> You can try this experiment:
> 
> a) cd /var/lib/shorewall-lite
> b) mv .iptables-restore-input bad-input
> c) shorewall-lite save
> d) diff -au bad-input .iptables-restore-input
> 
> What are the differences?
Sorry -- the name of the file in Shorewall-lite is restore-iptables, not 
.iptables-restore-input

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key



-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It''s the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

Tom Eastep wrote:
> Compare the output of ''shorewall-lite dump'' before and
after the restart.
it''s rather huge and very different (because of packet number 
differences it''s not easy to compare). it''d be better to
create some
kind of better dump which is easier to compare.
but i assume i find it (diff -u)!:

-   /proc/sys/net/ipv4/ip_forward = 0
+   /proc/sys/net/ipv4/ip_forward = 1

how can it be possible?
the strange there are other differences like:

-broadcast 213.253.216.128 dev eth1  proto kernel  scope link  src 
213.253.216.130
+xt_comment              5953  0
+xt_policy               7617  0


>> the only difference what i find in the sysinit script is -f option to 
>> shorewall but in /var/lib/shorewall-lite/ the file firewall and 
>> restore are the same.
> 
> The files /var/lib/shorewall-lite/firewall and 
> /var/lib/shorewall/restore are supposed to be the same if you have done 
> a ''shorewall-lite save''.
> 
>> if i comment out the -f option then it''s working without any 
>> workaround (or this is the workaround).
>> so what can be the reason?
>>
> 
> The file /var/lib/shorewall-lite/.iptables-restore-input is probably 
> wrong. But since I can''t see it, I can''t tell you what is
wrong with it.
> 
> You can try this experiment:
> 
> a) cd /var/lib/shorewall-lite
> b) mv .iptables-restore-input bad-input
> c) shorewall-lite save
> d) diff -au bad-input .iptables-restore-input
i try
cd /var/lib/shorewall-lite/
rm -rf * .??*
after that i reload from the central server and reboot
but still not working:-(

-- 
   Levente                               "Si vis pacem para bellum!"

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It''s the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

Farkas Levente wrote:
> Tom Eastep wrote:
>> Compare the output of ''shorewall-lite dump'' before
and after the restart.
> 
> it''s rather huge and very different (because of packet number 
> differences it''s not easy to compare). it''d be better to
create some
> kind of better dump which is easier to compare.
> but i assume i find it (diff -u)!:
> 
> -   /proc/sys/net/ipv4/ip_forward = 0
> +   /proc/sys/net/ipv4/ip_forward = 1
> 
> how can it be possible?
I assume that some other init script is turning it off after Shorewall-lite 
turns it on. What do you have in /etc/sysctl.conf?

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key



-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It''s the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

Tom Eastep wrote:
> Farkas Levente wrote:
>> Tom Eastep wrote:
>>> Compare the output of ''shorewall-lite dump''
before and after the
>>> restart.
>>
>> it''s rather huge and very different (because of packet number 
>> differences it''s not easy to compare). it''d be better
to create some
>> kind of better dump which is easier to compare.
>> but i assume i find it (diff -u)!:
>>
>> -   /proc/sys/net/ipv4/ip_forward = 0
>> +   /proc/sys/net/ipv4/ip_forward = 1
>>
>> how can it be possible?
> 
> I assume that some other init script is turning it off after 
> Shorewall-lite turns it on. What do you have in /etc/sysctl.conf?
net.ipv4.ip_forward = 0
but this is for years and it was working until now:-(
and there are the scripts after shorewall it doesn''t seem to be any of 
them can be:

S25shorewall-lite
S26apmd
S26lm_sensors
S26ups
S44acpid
S50snmpd
S50yum-cron
S55arpwatch
S55sshd
S58ntpd
S61clamd
S78spamassassin
S79amavisd
S79postgrey
S80postfix
S81ulogd
S85gpm
S85httpd
S90crond
S93ntop
S95anacron
S95atd
S98haldaemon
S99local
S99smartd

what''s more i modify the sysinit script like this:
case "$command" in
     start)
cat /proc/sys/net/ipv4/ip_forward >>/tmp/out
         #exec /sbin/shorewall-lite $OPTIONS $@
         /sbin/shorewall-lite $OPTIONS $@
cat /proc/sys/net/ipv4/ip_forward >>/tmp/out
and i''ve got two 0 0 !!!


-- 
   Levente                               "Si vis pacem para bellum!"

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It''s the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

Farkas Levente wrote:
> what''s more i modify the sysinit script like this:
> case "$command" in
>      start)
> cat /proc/sys/net/ipv4/ip_forward >>/tmp/out
>          #exec /sbin/shorewall-lite $OPTIONS $@
>          /sbin/shorewall-lite $OPTIONS $@
> cat /proc/sys/net/ipv4/ip_forward >>/tmp/out
> and i''ve got two 0 0 !!!
So what do you want me to do? I personally run Shorewall-lite and I
don''t
have this problem. So you have all the information that is needed to solve 
the problem; I don''t.

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key



-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It''s the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

Tom Eastep wrote:
> Farkas Levente wrote:
> 
>> what''s more i modify the sysinit script like this:
>> case "$command" in
>>      start)
>> cat /proc/sys/net/ipv4/ip_forward >>/tmp/out
>>          #exec /sbin/shorewall-lite $OPTIONS $@
>>          /sbin/shorewall-lite $OPTIONS $@
>> cat /proc/sys/net/ipv4/ip_forward >>/tmp/out
>> and i''ve got two 0 0 !!!
> 
> So what do you want me to do? I personally run Shorewall-lite and I 
> don''t have this problem. So you have all the information that is
needed
> to solve the problem; I don''t.
My apologies -- you actually gave me all the clues I needed to solve the problem

The bug has nothing to do with Shorewall-lite but rather is a result of the 
change that I made in 4.0.11 to defer setting up ip forwarding until after 
the rules are in place. I missed the case where the command is
''restore''
(which is what occurs when -f is specified to ''start'' and
there is a saved
config).

Patch attached.

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key



-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It''s the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

Tom Eastep wrote:
> The bug has nothing to do with Shorewall-lite but rather is a result of 
> the change that I made in 4.0.11 to defer setting up ip forwarding until 
> after the rules are in place. I missed the case where the command is 
> ''restore'' (which is what occurs when -f is specified to
''start'' and
> there is a saved config).
> 
> Patch attached.
I''ve also uploaded Shorewall-perl 4.0.11.2 which corrects the problem.

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key



-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It''s the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

Tom Eastep wrote:
> Tom Eastep wrote:
> 
>> The bug has nothing to do with Shorewall-lite but rather is a result 
>> of the change that I made in 4.0.11 to defer setting up ip forwarding 
>> until after the rules are in place. I missed the case where the 
>> command is ''restore'' (which is what occurs when -f is
specified to
>> ''start'' and there is a saved config).
>>
>> Patch attached.
> 
> I''ve also uploaded Shorewall-perl 4.0.11.2 which corrects the
problem.
And Shorewall-perl-4.0.12.1.

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key



-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It''s the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

Tom Eastep wrote:
> Tom Eastep wrote:
>> Tom Eastep wrote:
>>
>>> The bug has nothing to do with Shorewall-lite but rather is a
result
>>> of the change that I made in 4.0.11 to defer setting up ip
forwarding
>>> until after the rules are in place. I missed the case where the 
>>> command is ''restore'' (which is what occurs when
-f is specified to
>>> ''start'' and there is a saved config).
>>>
>>> Patch attached.
>>
>> I''ve also uploaded Shorewall-perl 4.0.11.2 which corrects the
problem.
> 
> And Shorewall-perl-4.0.12.1.
why not you release 4.0.13? versions are cheap.
anyway even it''s a small bug and the fix is very little imho
it''s a
serious bug. since anyone who update to 4.0.11 and later sometime reboot 
his firewall (what''s more do it remotely) cause that it''s stop
working.
and it''s hard to find the reason (i learn it in the hard way). and 
currently the latest version in fedora, redhat and centos (epel) is 
4.0.11 which makes thing worst.

-- 
   Levente                               "Si vis pacem para bellum!"

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It''s the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php

Farkas Levente wrote:
> 
> why not you release 4.0.13? versions are cheap.
That''s a pretty foolish thing for you to say, given that you have no
way of
knowing what is involved in producing a full Shorewall release. I suppose 
all things are cheap if you aren''t the one who has to do them.
> anyway even it''s a small bug and the fix is very little imho
it''s a
> serious bug. since anyone who update to 4.0.11 and later sometime reboot 
> his firewall (what''s more do it remotely) cause that it''s
stop working.
> and it''s hard to find the reason (i learn it in the hard way).
Only if they are running Shorewall-lite and only if they have done a 
''shorewall-lite save'' and only if their /etc/sysctl.conf
doesn''t enable
forwarding, and only if they don''t upgrade to 4.0.12 (which has the
updated
Shorewall-perl 4.0.12.1 package on all download sites).

  and
> currently the latest version in fedora, redhat and centos (epel) is 
> 4.0.11 which makes thing worst.
> 
The current version in fedora, redhat and centos is not really relevant to 
the question of whether I do a bug-fix release of Shorewall-perl or a full 
release of all four packages. Note that there is also a 
Shorewall-perl-4.0.11.2 which fixes the bug.

As a final note, Shorewall-lite 4.0.13 will remove the -f option from 
''/etc/init.d/shorewall-lite start'' to make it consistent with
Shorewall.

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key



-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It''s the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php