Shorewall users - Sep 2002 - shorewall restart disconnects me

Hi,

I have a rh72 (shorewall 1.3.2) running a six NIC firewall (4 active at the
moment). I worried about putting the other ttwo online because of the following
:

9 times out of 10 when I do a "service shorewall restart", I loose my
ssh connection to the box. Lucky me, there is a failover firewall that picks up
withing 5 seconds. But everytime this happens, I have to call the site and have
someone either reboot the main firewall or issue the service shorewall stop,
service shorewall restart command by hand (depending on who I get a hold off).

Any reason that would happen ?

I have shorewall on other smaller machines, works fine.

My rules files has about 100 active lines

I''m working on setting up shorewall on a 6 active NICs with a rules
file that will have at lease 300 to 400 rules.

How can I get around this problem ?

Frantzcy
-- 

Unreachability is bliss

On Wednesday 11 September 2002 16:00, Frantzcy Paisible
wrote:
> Hi,
>
> I have a rh72 (shorewall 1.3.2) running a six NIC firewall (4 active at the
> moment). I worried about putting the other ttwo online because of the
> following :
>
> 9 times out of 10 when I do a "service shorewall restart", I
loose my ssh
Hi,

I''m not sure what happens here but I''ve never seen such
behaviour. My only=20
guess is that using "service shorewall restart" is not the way to do
restart=20
of a "running" shorewall. I always use "shorewall restart"
which uses=20
/sbin/shorewall to reinitialize the firewall. Can you try whether this helps?

Simon
> connection to the box. Lucky me, there is a failover firewall that picks up
> withing 5 seconds. But everytime this happens, I have to call the site and
> have someone either reboot the main firewall or issue the service shorewall
> stop, service shorewall restart command by hand (depending on who I get a
> hold off).
>
> Any reason that would happen ?
>
> I have shorewall on other smaller machines, works fine.
>
> My rules files has about 100 active lines
>
> I''m working on setting up shorewall on a 6 active NICs with a
rules file
> that will have at lease 300 to 400 rules.
>
> How can I get around this problem ?
>
> Frantzcy

I have to disagree with the previous post. "service shorewall restart"
is
exactly the same as shorewall restart, it''s just a RedHat thing. It 
shouldn''t make a difference what command you use.

One option to consider is to add the routestopped option on the interface 
you are connecting from (in the interfaces file). This may have security 
implications, defaulting to ACCEPTing all connections on that interface 
until shorewall comes back up. Whether this is an option for you really 
depends on the rest of your setup. If you can do this on an interface that 
isn''t normally used, that''s even better.
At least put routestopped on an internal interface, so that when the backup 
firewall kicks in, you can connect from another internal machine and
don''t
need to call someone for physical access.

If this is not an option, it''s probably not coming back up because the
ssh
shell you are running the command on terminates, killing the shell, and 
then killing the shorewall command you just issued halfway through the 
restart. I''ve had success using a subshell. Just type:
# (service shorewall restart)
With the parentheses around it. This starts a new subshell, which should 
not exit when your ssh session exits. I''ve found that when I do this,
the
ssh session momentarily becomes unresponsive, but the rules usually come 
back up before the ssh session terminates. And - even when ssh does 
terminate (very likely with you - if you have 6 interfaces then shorewall 
is probably going to take some time to restart, longer than the ssh 
timeout) - then the shorewall command continues to run in the subshell and 
brings the firewall back up, so I can connect a few minutes later.

Your best bet - a combination of both. This way it should fail much less 
often, and if it does, you have an alternate means to connect to finish 
restarting it.

~Jonathan

PS - if you change the rules, be sure to use shorewall try instead of 
service shorewall restart.

--On Wednesday, September 11, 2002 4:15 PM +0200 Simon Matter 
<simon.matter@ch.sauter-bc.com> wrote:
> On Wednesday 11 September 2002 16:00, Frantzcy Paisible wrote:
>> Hi,
>>
>> I have a rh72 (shorewall 1.3.2) running a six NIC firewall (4 active at
>> the moment). I worried about putting the other ttwo online because of
the
>> following :
>>
>> 9 times out of 10 when I do a "service shorewall restart", I
loose my ssh
>
> Hi,
>
> I''m not sure what happens here but I''ve never seen such
behaviour. My
> only  guess is that using "service shorewall restart" is not the
way to
> do restart  of a "running" shorewall. I always use
"shorewall restart"
> which uses  /sbin/shorewall to reinitialize the firewall. Can you try
> whether this helps?
>
> Simon
>
>> connection to the box. Lucky me, there is a failover firewall that
picks
>> up withing 5 seconds. But everytime this happens, I have to call the
>> site and have someone either reboot the main firewall or issue the
>> service shorewall stop, service shorewall restart command by hand
>> (depending on who I get a hold off).
>>
>> Any reason that would happen ?
>>
>> I have shorewall on other smaller machines, works fine.
>>
>> My rules files has about 100 active lines
>>
>> I''m working on setting up shorewall on a 6 active NICs with a
rules file
>> that will have at lease 300 to 400 rules.
>>
>> How can I get around this problem ?
>>
>> Frantzcy
> _______________________________________________
> Shorewall-users mailing list
> Shorewall-users@shorewall.net
> http://www.shorewall.net/mailman/listinfo/shorewall-users

On Wednesday 11 September 2002 07:48 am, Jonathan Manning
wrote:
> I have to disagree with the previous post. "service shorewall
restart" is
> exactly the same as shorewall restart, it''s just a RedHat thing.
It
> shouldn''t make a difference what command you use.
>
> One option to consider is to add the routestopped option on the interface
> you are connecting from (in the interfaces file).
Use of the interfaces file for specifying "routestopped" is deprecated
-- the
/etc/shorewall/routestopped is the preferred method.

-Tom
-- 
Tom Eastep    \ Shorewall - iptables made easy
AIM: tmeastep  \ http://www.shorewall.net
ICQ: #60745924  \ teastep@shorewall.net

Hi,

 First, guys, thanx for taking the time...

 Well, yes, server shorewall restart and /sbin/shorewall restart, same thing.
I do have routestopped on the interface Iam connecting from. Beleive me, I
learned the hard way the true meaning of routestopped, specialy with unforgiving
errors you can make in the config.

When, I lose the connection, the firewall doesn''t respond to
anything... quite odd

(service shorewall restart) &  did help, but I realy don''t wanna
have to do it...
Simply because, once I do add more rules (300-400) it my system gonna be down
for 5 minutes everytome I restart the firewall ?

ahhh, shorewall try /etc/shorewall,  the commands from heaven... :)

Frantzcy

On Wed, 11 Sep 2002 10:48:53 -0400,Jonathan Manning
<jmanning@alisa-jon.net> wrote:
> I have to disagree with the previous post. "service shorewall
restart" is
> exactly the same as shorewall restart, it''s just a RedHat thing.
It
> shouldn''t make a difference what command you use.
> 
> One option to consider is to add the routestopped option on the interface 
> you are connecting from (in the interfaces file). This may have security 
> implications, defaulting to ACCEPTing all connections on that interface 
> until shorewall comes back up. Whether this is an option for you really 
> depends on the rest of your setup. If you can do this on an interface that 
> isn''t normally used, that''s even better.
> At least put routestopped on an internal interface, so that when the backup
> firewall kicks in, you can connect from another internal machine and
don''t
> need to call someone for physical access.
> 
> If this is not an option, it''s probably not coming back up because
the ssh
> shell you are running the command on terminates, killing the shell, and 
> then killing the shorewall command you just issued halfway through the 
> restart. I''ve had success using a subshell. Just type:
> # (service shorewall restart)
> With the parentheses around it. This starts a new subshell, which should 
> not exit when your ssh session exits. I''ve found that when I do
this, the
> ssh session momentarily becomes unresponsive, but the rules usually come 
> back up before the ssh session terminates. And - even when ssh does 
> terminate (very likely with you - if you have 6 interfaces then shorewall 
> is probably going to take some time to restart, longer than the ssh 
> timeout) - then the shorewall command continues to run in the subshell and 
> brings the firewall back up, so I can connect a few minutes later.
> 
> Your best bet - a combination of both. This way it should fail much less 
> often, and if it does, you have an alternate means to connect to finish 
> restarting it.
> 
> ~Jonathan
> 
> PS - if you change the rules, be sure to use shorewall try instead of 
> service shorewall restart.
> 
> --On Wednesday, September 11, 2002 4:15 PM +0200 Simon Matter 
> <simon.matter@ch.sauter-bc.com> wrote:
> 
> > On Wednesday 11 September 2002 16:00, Frantzcy Paisible wrote:
> >> Hi,
> >>
> >> I have a rh72 (shorewall 1.3.2) running a six NIC firewall (4
active at
> >> the moment). I worried about putting the other ttwo online because
of the
> >> following :
> >>
> >> 9 times out of 10 when I do a "service shorewall
restart", I loose my ssh
> >
> > Hi,
> >
> > I''m not sure what happens here but I''ve never seen
such behaviour. My
> > only  guess is that using "service shorewall restart" is not
the way to
> > do restart  of a "running" shorewall. I always use
"shorewall restart"
> > which uses  /sbin/shorewall to reinitialize the firewall. Can you try
> > whether this helps?
> >
> > Simon
> >
> >> connection to the box. Lucky me, there is a failover firewall that
picks
> >> up withing 5 seconds. But everytime this happens, I have to call
the
> >> site and have someone either reboot the main firewall or issue the
> >> service shorewall stop, service shorewall restart command by hand
> >> (depending on who I get a hold off).
> >>
> >> Any reason that would happen ?
> >>
> >> I have shorewall on other smaller machines, works fine.
> >>
> >> My rules files has about 100 active lines
> >>
> >> I''m working on setting up shorewall on a 6 active NICs
with a rules file
> >> that will have at lease 300 to 400 rules.
> >>
> >> How can I get around this problem ?
> >>
> >> Frantzcy
> > _______________________________________________
> > Shorewall-users mailing list
> > Shorewall-users@shorewall.net
> > http://www.shorewall.net/mailman/listinfo/shorewall-users
> 

-- 
http://www.paisible.qc.ca

On Wednesday 11 September 2002 09:08 am, Frantzcy Paisible
wrote:
> Hi,
>
>  First, guys, thanx for taking the time...
>
>  Well, yes, server shorewall restart and /sbin/shorewall restart, same
> thing. I do have routestopped on the interface Iam connecting from. Beleive
> me, I learned the hard way the true meaning of routestopped, specialy with
> unforgiving errors you can make in the config.
>
> When, I lose the connection, the firewall doesn''t respond to
anything...
> quite odd
What does "shorewall status" show? I suspect that the
"restart" is failing but
until we know, reports of "firewall doesn''t respond to
anything" are useless.

>
> (service shorewall restart) &  did help, but I realy don''t
wanna have to do
> it... Simply because, once I do add more rules (300-400) it my system gonna
> be down for 5 minutes everytome I restart the firewall ?
To change the ruleset, you are going to have to restart the firewall -- there 
is no way around that.
>
> ahhh, shorewall try /etc/shorewall,  the commands from heaven... :)
>
NO !!!!!!!!!!!!!!!!!!!!!! DO NOT "shorewall try /etc/shorewall". You
want to:

mkdir /etc/test

cp /etc/shorewall/<files that you are going to modify> /etc/test
<modify files in /etc/test>
shorewall try /etc/test

-Tom
-- 
Tom Eastep    \ Shorewall - iptables made easy
AIM: tmeastep  \ http://www.shorewall.net
ICQ: #60745924  \ teastep@shorewall.net

Tom Eastep wrote:

> What does "shorewall status" show? I suspect that the
"restart" is failing but
> until we know, reports of "firewall doesn''t respond to
anything" are useless.
I have seen this problem too. I have a firewall with 18 interfaces and I 
can confirm that shorewall has serious performance problems.
I had to remove all zone2zone REJECT rules from policy to get firewall 
startup time down from 2 minutes. And setup is simple, there are only 
140 rule lines on rules file. Now it takes more than one minute to 
startup. And when shorewall restart is going on, everything is stopped. 
There is no network traffic when restart is going on.

Is there any possibility to change shorewall so that:

shorewall restart won''t stop network-traffic

or

Is there possibility to change logic so that policy and rules between 
two zones are processed, activated, then next zone-pair is processed?

There really is disturbing performance problem.

Firewall I use in this case is pentium III over 500 MHz and 512MB of ram 
so it''s not _that_ slow.

-- 
Tuomo Soini <tis@foobar.fi>
http://tis.foobar.fi/

> 
> What does "shorewall status" show? I suspect that the
"restart" is failing but
> until we know, reports of "firewall doesn''t respond to
anything" are useless.
> 
> 
shorewall status give me the listing of my rules, about 650 lines.
When i guet kicked out, I''m already out can''T do the status
command there
> >
> > ahhh, shorewall try /etc/shorewall,  the commands from heaven... :)
> >
> 
> NO !!!!!!!!!!!!!!!!!!!!!! DO NOT "shorewall try /etc/shorewall".
You want to:
> 
Hahah! I notice, jst got kicked out again...


Frantzcy


-- 

Unreachability is bliss

On Wednesday 11 September 2002 12:00 pm, Frantzcy Paisible
wrote:
> > What does "shorewall status" show? I suspect that the
"restart" is
> > failing but until we know, reports of "firewall doesn''t
respond to
> > anything" are useless.
>
> shorewall status give me the listing of my rules, about 650 lines.
> When i guet kicked out, I''m already out can''T do the
status command there
Well, if you can''t see the rules we sure can''t -- until we
know what is
happening, there isn''t much I can do to help you. Again, I suspect that
the
"restart" is failing for some reason but without knowing, I
don''t know what I
can tell you...
>
> > > ahhh, shorewall try /etc/shorewall,  the commands from heaven...
:)
> >
> > NO !!!!!!!!!!!!!!!!!!!!!! DO NOT "shorewall try
/etc/shorewall". You want
> > to:
>
> Hahah! I notice, jst got kicked out again...
>
>
> Frantzcy
-- 
Tom Eastep    \ Shorewall - iptables made easy
AIM: tmeastep  \ http://www.shorewall.net
ICQ: #60745924  \ teastep@shorewall.net

> > shorewall status give me the listing of my rules, about 650 lines.
> > When i guet kicked out, I''m already out can''T do the
status command there
> 
> Well, if you can''t see the rules we sure can''t -- until
we know what is
> happening, there isn''t much I can do to help you. Again, I suspect
that the
> "restart" is failing for some reason but without knowing, I
don''t know what I
> can tell you...
> 
Well if I reboot the server everything goes back to normal. But hey! this is now
windows machine, so I can''t accept this method os problem solving...


Even worst... after service shorewall restart, not all my rules are active the
way they should be..., again, after reboot, everything ok...

Frantzcy
-- 

Unreachability is bliss

On Wednesday 11 September 2002 11:57 am, Tuomo Soini
wrote:
> Tom Eastep wrote:
> > What does "shorewall status" show? I suspect that the
"restart" is
> > failing but until we know, reports of "firewall doesn''t
respond to
> > anything" are useless.
>
> I have seen this problem too. I have a firewall with 18 interfaces and I
> can confirm that shorewall has serious performance problems.
> I had to remove all zone2zone REJECT rules from policy to get firewall
> startup time down from 2 minutes. And setup is simple, there are only
> 140 rule lines on rules file. Now it takes more than one minute to
> startup. And when shorewall restart is going on, everything is stopped.
> There is no network traffic when restart is going on.
Not true -- Shorewall enables existing connections while it is restarting -- 
it just doesn''t allow any new connections.
>
> Is there any possibility to change shorewall so that:
>
> shorewall restart won''t stop network-traffic
>
> or
>
> Is there possibility to change logic so that policy and rules between
> two zones are processed, activated, then next zone-pair is processed?
It is possible to speed up restart although I would probably have to rewrite 
Shorewall in C. That way, the program could talk directly to NetFilter 
(through a library) and wouldn''t have to run iptables for each
NetFilter
change. I''m reluctant to do that but will at least consider it for
Shorewall
2.0. The problem is that unless Shorewall starts with a "clean"
Netfilter
environment, it has to analyze the current environment then try to determine 
what is different about that environment and the new one.  It then has to 
change the current environment to match the new environment without allowing 
any unwanted connections or rejecting any allowed connections. This is not an 
easy problem.

And no -- there is no time frame for 2.0.
>
> There really is disturbing performance problem.
>
> Firewall I use in this case is pentium III over 500 MHz and 512MB of ram
> so it''s not _that_ slow.
Interesting -- my setup has 41 rules, PII 266Hhz, 128MB ram and restart takes 
20 seconds. Does the processing of the rules file take the majority of the 
time during restart?

-Tom
-- 
Tom Eastep    \ Shorewall - iptables made easy
AIM: tmeastep  \ http://www.shorewall.net
ICQ: #60745924  \ teastep@shorewall.net

On Wednesday 11 September 2002 12:55 pm, Frantzcy Paisible
wrote:
> > > shorewall status give me the listing of my rules, about 650
lines.
> > > When i guet kicked out, I''m already out can''T
do the status command
> > > there
> >
> > Well, if you can''t see the rules we sure can''t --
until we know what is
> > happening, there isn''t much I can do to help you. Again, I
suspect that
> > the "restart" is failing for some reason but without
knowing, I don''t
> > know what I can tell you...
>
> Well if I reboot the server everything goes back to normal. But hey! this
> is now windows machine, so I can''t accept this method os problem
solving...
>
>
> Even worst... after service shorewall restart, not all my rules are active
> the way they should be..., again, after reboot, everything ok...
Then please "shorewall debug restart 2> /tmp/trace" from now on --
at least
when it dies, you can send me the /tmp/trace file. I''ve got to have
SOMETHING
to go on besides "rebooting fixes it"....

-Tom
-- 
Tom Eastep    \ Shorewall - iptables made easy
AIM: tmeastep  \ http://www.shorewall.net
ICQ: #60745924  \ teastep@shorewall.net

Yeah the reinistalisation time is too long, the more rules and interaces you
have, the more time it takes to reinitiate.

But incidentaly, the mode rules and interfaces you have, the more critical the
firewall becomes.

And in my case I have a failover firewall running heartbeat, and if
communications between the two firewalls stops for too long, the other one picks
up the IPs.

What can be done to fix this ?

Frantzcy


On Wed, 11 Sep 2002 21:57:47 +0300,Tuomo Soini <tis@foobar.fi> wrote:
> Tom Eastep wrote:
> 
> 
> > What does "shorewall status" show? I suspect that the
"restart" is failing but
> > until we know, reports of "firewall doesn''t respond to
anything" are useless.
> 
> I have seen this problem too. I have a firewall with 18 interfaces and I 
> can confirm that shorewall has serious performance problems.
> I had to remove all zone2zone REJECT rules from policy to get firewall 
> startup time down from 2 minutes. And setup is simple, there are only 
> 140 rule lines on rules file. Now it takes more than one minute to 
> startup. And when shorewall restart is going on, everything is stopped. 
> There is no network traffic when restart is going on.
> 
> Is there any possibility to change shorewall so that:
> 
> shorewall restart won''t stop network-traffic
> 
> or
> 
> Is there possibility to change logic so that policy and rules between 
> two zones are processed, activated, then next zone-pair is processed?
> 
> There really is disturbing performance problem.
> 
> Firewall I use in this case is pentium III over 500 MHz and 512MB of ram 
> so it''s not _that_ slow.
> 
> -- 
> Tuomo Soini <tis@foobar.fi>
> http://tis.foobar.fi/
> 
> _______________________________________________
> Shorewall-users mailing list
> Shorewall-users@shorewall.net
> http://www.shorewall.net/mailman/listinfo/shorewall-users

-- 

Unreachability is bliss

> > I had to remove all zone2zone REJECT rules from policy to get firewall
> > startup time down from 2 minutes. And setup is simple, there are only
> > 140 rule lines on rules file. Now it takes more than one minute to
> > startup. And when shorewall restart is going on, everything is
stopped.
> > There is no network traffic when restart is going on.
> 
> Not true -- Shorewall enables existing connections while it is restarting
--
> it just doesn''t allow any new connections.
> 
> >

Ok, Is it possible to (!! security alert !!) allow connections in the
reinitiation, and stop them after when closing the policies ?




> >
> > Is there possibility to change logic so that policy and rules between
> > two zones are processed, activated, then next zone-pair is processed?
> 
> It is possible to speed up restart although I would probably have to
rewrite
> Shorewall in C. That way, the program could talk directly to NetFilter 
> (through a library) and wouldn''t have to run iptables for each
NetFilter
> change. I''m reluctant to do that but will at least consider it for
Shorewall
> 2.0. The problem is that unless Shorewall starts with a "clean"
Netfilter
> environment, it has to analyze the current environment then try to
determine
> what is different about that environment and the new one.  It then has to 
> change the current environment to match the new environment without
allowing
> any unwanted connections or rejecting any allowed connections. This is not
an
> easy problem.

Would if be possible to go at it table by table, say clear net2dmz, add net2dmz,
clear dmz2loc , add dmz2loc ,, etc...
??


> 
> And no -- there is no time frame for 2.0.
> 

You saw us comming a mile away! lol!

> > Firewall I use in this case is pentium III over 500 MHz and 512MB of
ram
> > so it''s not _that_ slow.
> 
> Interesting -- my setup has 41 rules, PII 266Hhz, 128MB ram and restart
takes
> 20 seconds. Does the processing of the rules file take the majority of the 
> time during restart?
> 

I get 35-45 seconds on a PIII 1Ghz 256Mb ram (100 line rules file, 4 interfaces,
7 zones)

am
-- 

Unreachability is bliss

Tom Eastep wrote:
> It is possible to speed up restart although I would probably have to
rewrite
> Shorewall in C. That way, the program could talk directly to NetFilter 
> (through a library) and wouldn''t have to run iptables for each
NetFilter
> change. I''m reluctant to do that but will at least consider it for
Shorewall
> 2.0. The problem is that unless Shorewall starts with a "clean"
Netfilter
> environment, it has to analyze the current environment then try to
determine
> what is different about that environment and the new one.  It then has to 
> change the current environment to match the new environment without
allowing
> any unwanted connections or rejecting any allowed connections. This is not
an
> easy problem.
Or save information about last restart into simple text-based database 
about last run off shorewall? Names of "chains" and correlation
between
them? And if correlation and rule names in new run are same, just flush 
rule, Inser DROP at start, add rules for this chain, remove DROP and 
continue to next chain.

Sorry, I didn''t have time to actually check how shorewall does this 
currently.
> And no -- there is no time frame for 2.0.
I can understand that.
>>Firewall I use in this case is pentium III over 500 MHz and 512MB of ram
>>so it''s not _that_ slow.
> 
> 
> Interesting -- my setup has 41 rules, PII 266Hhz, 128MB ram and restart
takes
> 20 seconds. Does the processing of the rules file take the majority of the 
> time during restart?

To end of rules 30 seconds
Other stuff about 20 seconds
Activating rules 30 seconds

And as I said. I had to let REJECTs to all2all rule or processing policy 
would have taken more than 60 seconds...

-- 
Tuomo Soini <tis@foobar.fi>
http://tis.foobar.fi/

On Wednesday 11 September 2002 02:32 pm, Tuomo Soini wrote:
>
> Or save information about last restart into simple text-based database
> about last run off shorewall? Names of "chains" and correlation
between
> them? And if correlation and rule names in new run are same, just flush
> rule, Inser DROP at start, add rules for this chain, remove DROP and
> continue to next chain.
Do you most often restart Shorewall simply to add or delete rules?
>
> To end of rules 30 seconds
> Other stuff about 20 seconds
> Activating rules 30 seconds
>
> And as I said. I had to let REJECTs to all2all rule or processing policy
> would have taken more than 60 seconds...
The policy stuff is slow because the policy file is read repeatedly -- I might 
be able to speed that up without totally destablizing the code...

-Tom
-- 
Tom Eastep    \ Shorewall - iptables made easy
AIM: tmeastep  \ http://www.shorewall.net
ICQ: #60745924  \ teastep@shorewall.net

Tom Eastep wrote:
> 
> Do you most often restart Shorewall simply to add or delete rules?
Yes. If there is better way, I haven''t found it. Other thing is adding 
zone but that is so rare that current startup time is not too much.
>>To end of rules 30 seconds
>>Other stuff about 20 seconds
>>Activating rules 30 seconds
>>
>>And as I said. I had to let REJECTs to all2all rule or processing policy
>>would have taken more than 60 seconds...
> The policy stuff is slow because the policy file is read repeatedly -- I
might
> be able to speed that up without totally destablizing the code...
That would be ok. While I don''t have so many policy lines any more it 
takes about half of that 20 seconds I mention...

-- 
Tuomo Soini <tis@foobar.fi>
http://tis.foobar.fi/