Shorewall users - Dec 2005 - Shorewall stops after about 36 hours

I don''t know what is happening, but every 36 hours or so I''ll
go to ssh into
my server and find that I can''t connect on any of the outside services.
I
then can connect into the box using a serial connection and find that
shorewall reports that it is not running and the iptables are in some kind
of default state which looks nothing like what I set it up with shorewall
as.  I can restart shorewall in any fashion and it returns to its normal
working state, but until then, the outside world can''t connect to it
(thank
god for serial connections).  Here''s some more details on the problem:

shorewall version: 3.0.2 (Fedora rpm install)

ip addr show:
1: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:11:11:57:a9:82 brd ff:ff:ff:ff:ff:ff
    inet 82.165.182.175/32 brd 82.165.182.175 scope global eth0
2: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo

ip route show:
10.255.255.1 dev eth0  scope link
169.254.0.0/16 dev eth0  scope link
default via 10.255.255.1 dev eth0

For a complete dump, see: http://www.nanovox.com/temp/ipdump.txt
(and ignore the actual allocations of ip addresses.  I''ve changed a few
octets for security reasons, but they remain consistant so that they would
make sense.)

Steven Kiehl wrote:
> I don''t know what is happening, but every 36 hours or so
I''ll go to ssh into
> my server and find that I can''t connect on any of the outside
services...
most likely a cronjob or something in your system is restarting the
network.. an putting the packet filter in a clean state...

as a **possible** workaround you can put

#! /bin/sh
/sbin/shorewall restart

in your distribution ifup-** network scripts..

Steven Kiehl wrote:
> I don''t know what is happening, but every 36 hours or so
I''ll go to ssh into
> my server and find that I can''t connect on any of the outside
services.
and use this like a mantra "Shorewall is not a daemon, so it can''t
stop
itself after 36 hours"

> (and ignore the actual allocations of ip addresses.  I''ve changed
a few
> octets for security reasons, but they remain consistant so that they would
> make sense.)
> 
sometimes if you modify that information, you render it useless.
and ip addresses aren''t secrets at all.

On 12/13/05, Cristian Rodriguez <judas_iscariote@shorewall.net>
wrote:
>
> Steven Kiehl wrote:
> > I don''t know what is happening, but every 36 hours or so
I''ll go to ssh
> into
> > my server and find that I can''t connect on any of the outside
services.
>
> and use this like a mantra "Shorewall is not a daemon, so it
can''t stop
> itself after 36 hours"

I''m aware of this.  It''s merely a fancy policy editor.  What I
mean by
shorewall stops every 36 hours, is that a "shorewall status" reports
that
shorewall is stopped when it normally says that shorewall is running.
> (and ignore the actual allocations of ip addresses.  I''ve changed
a few
> > octets for security reasons, but they remain consistant so that they
> would
> > make sense.)
> >
> sometimes if you modify that information, you render it useless.
> and ip addresses aren''t secrets at all.

True.

After a bit more digging, I found that the policy that was replacing the
shorewall policies is that of a saved iptables configuration.  I had saved
it in case I needed it as a reference.  But this doesn''t totally solve
my
problem.  I need to know what is loading this default policy and how I can
fix that.  Should I just set iptables config up to save the configuration on
shutdown?  I have serial access, so I guess that would be reasonable.  But
does anyone know why iptables would restart and load the saved policy
instead of the shorewall policy?  Or and I just being dumb, and everyone
running shorewall has IPTABLES_SAVE_ON_STOP and IPTABLES_SAVE_ON_RESTART set
to ''yes''?

On Tuesday 13 December 2005 19:28, Steven Kiehl wrote:
> Or and I just being dumb, and everyone
> running shorewall has IPTABLES_SAVE_ON_STOP and IPTABLES_SAVE_ON_RESTART
> set to ''yes''?
Everyone running Shorewall is expected to disable all other iptables
"stuff"
on their system.

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key

On Tuesday 13 December 2005 19:32, Tom Eastep wrote:
> On Tuesday 13 December 2005 19:28, Steven Kiehl wrote:
> > Or and I just being dumb, and everyone
> > running shorewall has IPTABLES_SAVE_ON_STOP and
IPTABLES_SAVE_ON_RESTART
> > set to ''yes''?
>
> Everyone running Shorewall is expected to disable all other iptables
> "stuff" on their system.
In your case, that means "chkconfig -del itpables". AFAIK,
that''s the default
on Fedora so it must have been enabled on your system at some point. I had to 
explicitly enable it when I added a REDIRECT rule for Squid (see 
http://www.shorewall.net/Shorewall_Squid_Usage.html).

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key

Sorry to bother you guys again, but iptables did it again and is back to its
default state for whatever strange reason.  I did a chkconfig --del iptables
and a chkconfig --del ip6tables last time, and things seemed fine until just
tonight.  I was logged into it all day, signed out and then a few hours
later now I try to check the web server and find that shorewall is back to
saying it''s stopped.

Right now I''m using the shorewall 3.0.2-1 fedora development rpm
install.
Do you think anything in that would cause it to stop after 48 hours? 
I''m
really getting frustrated with this thing, I''m either gonna have to
start
uninstalling everything until shorewall stops resetting, or make a cron task
to check the status every 5 minutes and restart it if it reverts to being
stopped.

I have a number of web-related and mail-related stuff running, but none of
which would mess with iptables.

Here''s a list of my services status messages:
anacron is stopped
atd (pid 1514) is running...
cpuspeed is stopped
crond (pid 1474) is running...
usage: /etc/init.d/freshclam start|stop
gpm is stopped
irqbalance is stopped
dbus-daemon (pid 1521) is running...
/etc/init.d/microcode_ctl: microcode device /dev/cpu/0/microcode
doesn''t
exist?
Usage: /etc/init.d/mysql start|stop|restart|reload
Server address not specified in /etc/sysconfig/netdump
netplugd is stopped
Configured devices:
lo eth0
Currently active devices:
eth0 lo
ntpd is stopped
Process accounting is disabled.
saslauthd is stopped
Shorewall is stopped
    State:Started (Tue Dec 13 23:12:09 EST 2005)
smartd dead but subsys locked
spamd (pid 6697 6696 6695 6694 6693 6647) is running...
sshd (pid 1380) is running...
syslogd (pid 1214) is running...
klogd (pid 1216) is running...
Xvnc is stopped
xfs (pid 1501) is running...
Nightly yum update is disabled.


As far as cron tasks go...

Daily:
logwatch    // watches log changes
makewhatis.cron    // manpage whatis database
anacron        // cron maintenance stuff
logrotate    // rotates logs (none of which touch iptables)
prelink        // I have no idea
rpm        // logs rpm database changes
slocate.cron    // updates locate database
spamupdate    // updates some of my spam filtering stuff
tmpwatch    // cleans up the tmp directory
yum.cron    // checks yum for updates
update_tmprsadh // updates the email certs every night

Hourly:
quotacheck    // ftp quota check

Any ideas?

On 12/13/05, Tom Eastep <teastep@shorewall.net>
wrote:
>
> On Tuesday 13 December 2005 19:32, Tom Eastep wrote:
> > On Tuesday 13 December 2005 19:28, Steven Kiehl wrote:
> > > Or and I just being dumb, and everyone
> > > running shorewall has IPTABLES_SAVE_ON_STOP and
> IPTABLES_SAVE_ON_RESTART
> > > set to ''yes''?
> >
> > Everyone running Shorewall is expected to disable all other iptables
> > "stuff" on their system.
>
> In your case, that means "chkconfig -del itpables". AFAIK,
that''s the
> default
> on Fedora so it must have been enabled on your system at some point. I had
> to
> explicitly enable it when I added a REDIRECT rule for Squid (see
> http://www.shorewall.net/Shorewall_Squid_Usage.html).
>
> -Tom
> --
> Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
> Shoreline,     \ http://shorewall.net
> Washington USA  \ teastep@shorewall.net
> PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key
>
>
>

On Thursday 15 December 2005 17:49, Steven Kiehl wrote:
>
> I have a number of web-related and mail-related stuff running, but none of
> which would mess with iptables.
>
What type of internet connection do you have? PPPoE?

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key

As far as I know it''s just have your basic LAN setup with dhcp and all
with
an Intel 82562EZ 10/100 Ethernet Controller.

On 12/15/05, Tom Eastep <teastep@shorewall.net>
wrote:
>
> On Thursday 15 December 2005 17:49, Steven Kiehl wrote:
>
> >
> > I have a number of web-related and mail-related stuff running, but
none
> of
> > which would mess with iptables.
> >
>
> What type of internet connection do you have? PPPoE?
>
> -Tom
> --
> Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
> Shoreline,     \ http://shorewall.net
> Washington USA  \ teastep@shorewall.net
> PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key
>
>
>

Still getting these restarts, but I''ve croned a 5-minute job to check
the
shorewall status, restart it if need be, and log a message saying when it
restarted.  So far it restarted on saturday afternoon at 12:30pm and this
morning at 6:30am.  That''s a 42 hour span which makes no sense. 
However,
here''s something interesting... At 6:28am this morning my dhcp renewed,
and
sure enough dhcp also renewed at 12:25pm on saturday. Perhaps something with
dhcp is resetting my iptables?  Any ideas what script might be resetting the
iptables after dhcp renews?

Also, how can I tell if dhcp is getting responses back properly after a dhcp
request?  Do I just assume it works because I don''t see any logs from
shorewall about port 68 being blocked?

I guess a cronjob to restart shorewall wasn''t a bad idea afterall now
that I
know what program is resetting my iptables.  Now I just need to get dhclient
to stop resetting the firewall.

On 12/15/05, Steven Kiehl <nanovox@gmail.com>
wrote:
>
> As far as I know it''s just have your basic LAN setup with dhcp and
all
> with an Intel 82562EZ 10/100 Ethernet Controller.
>
> On 12/15/05, Tom Eastep < teastep@shorewall.net> wrote:
> >
> > On Thursday 15 December 2005 17:49, Steven Kiehl wrote:
> >
> > >
> > > I have a number of web-related and mail-related stuff running,
but
> > none of
> > > which would mess with iptables.
> > >
> >
> > What type of internet connection do you have? PPPoE?
> >
> > -Tom
> > --
> > Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
> > Shoreline,     \ http://shorewall.net
> > Washington USA  \ teastep@shorewall.net
> > PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key
> >
> >
> >
>

On Monday 19 December 2005 07:37, Steven Kiehl wrote:
> Still getting these restarts, but I''ve croned a 5-minute job to
check the
> shorewall status, restart it if need be, and log a message saying when it
> restarted.  So far it restarted on saturday afternoon at 12:30pm and this
> morning at 6:30am.  That''s a 42 hour span which makes no sense. 
However,
> here''s something interesting... At 6:28am this morning my dhcp
renewed, and
> sure enough dhcp also renewed at 12:25pm on saturday. Perhaps something
> with dhcp is resetting my iptables?  Any ideas what script might be
> resetting the iptables after dhcp renews?
"man dhclient" -- dhcp clients can run scripts when certain events
occur but I
don''t know the details about dhclient and I don''t seem to have
it installed
on any of my systems..
>
> Also, how can I tell if dhcp is getting responses back properly after a
> dhcp request?  Do I just assume it works because I don''t see any
logs from
> shorewall about port 68 being blocked?
If you''ve specified ''dhcp'' on the interface, then
Shorewall won''t block (or
log) DHCP traffic. 
>
> I guess a cronjob to restart shorewall wasn''t a bad idea afterall
now that
> I know what program is resetting my iptables.  Now I just need to get
> dhclient to stop resetting the firewall.
Good Luck,
-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key