Shorewall devel - May 2005 - shorewall startup speed - an idea

Dear All,

I think I have a useful idea for how shorewall startup could be speeded 
up in a more automatic manner. Apologies if this is daft, but I think it 
might work....

Motivation: not all users understand the intricacies of shoreall beyond 
using the distro setup tool. [And on this particular laptop, shorewall 
takes 15 seconds during boot.]

I have already read this (about shorewall save/restore):
http://www.shorewall.net/FAQ.htm#faq34
but the problem is that this is not automatic - it requires intervention 
from the user. What would be nice would be something akin to the way 
"make" handles compiles, i.e. if nothing has changed, the shorewall 
config isn''t re-parsed.

Here is a possible way it might work:

1)After a successful shorewall startup, shorewall always saves its state 
to: /var/lib/shorewall/last.state


2)The next time shorewall starts, it checks:
   i)Does /var/lib/shorewall/restore exist?  If so, use that.

   ii)Otherwise, does /var/lib/shorewall/last.state exist? If so:
    Is /var/lib/shorewall/last.state newer than the all the files
    in /etc/shorewall?  If so: use /var/lib/shorewall/last.state

   iii)Otherwise, re-parse all the rules in /etc/shorewall/
    (as usual.)


The advantages of doing this are:

i)It doesn''t interfere with the existing setup.

ii)It is transparent to the user

iii)It provides a significant speedup in the common case of the 
configuration being unchanged (which is quite important on a laptop, 
since these frequently get restarted)

iv)I don''t think that there is any security risk here, because the only
test is comparing 2 filemtime(s).


I hope this suggestion is a useful one. Thank you very much for your 
work - it''s much appreciated!


Best wishes

Richard


-- 
rn214@hermes.cam.ac.uk  **  http://www.richardneill.org
Richard Neill, Trinity College, Cambridge, CB21TQ, U.K.

Hello Richard,

Richard Neill wrote:
> 
>   ii)Otherwise, does /var/lib/shorewall/last.state exist? If so:
>    Is /var/lib/shorewall/last.state newer than the all the files
>    in /etc/shorewall?  If so: use /var/lib/shorewall/last.state
> 
This needs to consider:

a) CONFIG_PATH -- config files can be scattered over lots of directories,
not just /etc/shorewall.
b) Extension scripts -- there can be an extension script for any chain in
the old configuration. So if you change one of them, you would expect that
"shorewall restart" would reprocess all of the config files, right?
That
means that you have to actually KNOW the names of the files that you need to
check; they aren''t just limited to the released Shoreall configuration
files.
c) I also worry about people starting a configuration that doesn''t work
and
having Shorewall stupidly save that configuration and then continue to
slavishly reinstate it every time the user attempts to correct the situation
by typing "shorewall restart" or (as most naive users do) rebooting.
It''s
important that the state _only be saved when it is a valid state_. Lorenzo
Martignoni (the Shorewall Debian maintainer) is doing some work that may
help this. We''ll see.
d) Unfortunately, this isn''t easy. I''ve thought of trying to
do something
like this but I really don''t want to spend the cycles it would require
to do
it right just so that people don''t have to type "shorewall
save" at a root
shell prompt. And the user KNOWS when the configuration is right or wrong --
Shorewall doesn''t.

Still, thanks for thinking about improving Shorewall. I appreciate it. And
after Lorenzo get his "safe-restart" command working, we can probably
integrate some of your ideas into his work.

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key

2005/5/12, Tom Eastep <teastep@shorewall.net>:
> 
> Still, thanks for thinking about improving Shorewall. I appreciate it. And
> after Lorenzo get his "safe-restart" command working, we can
probably
> integrate some of your ideas into his work.
> 
Im using ash,And it takes only 2 o 3 seconds to start,I dont think
that is too much time.
Changing the default shell is a good solution,but it generates another
dependency.(ash is shipped on all  major distros...)

Cristian Rodriguez wrote:
> 2005/5/12, Tom Eastep <teastep@shorewall.net>:
> 
>>Still, thanks for thinking about improving Shorewall. I appreciate it.
And
>>after Lorenzo get his "safe-restart" command working, we can
probably
>>integrate some of your ideas into his work.
>>
> 
> Im using ash,And it takes only 2 o 3 seconds to start,I dont think
> that is too much time.
> Changing the default shell is a good solution,but it generates another
> dependency.(ash is shipped on all  major distros...)
I use dash under Debian -- it also speeds up "shorewall [re]start"
substantially.

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key

>>
>>   ii)Otherwise, does /var/lib/shorewall/last.state exist? If so:
>>    Is /var/lib/shorewall/last.state newer than the all the files
>>    in /etc/shorewall?  If so: use /var/lib/shorewall/last.state
>> 
 >This needs to consider:
 >
 >a) CONFIG_PATH -- config files can be scattered over lots of 
 >directories, not just /etc/shorewall.

Yes. But presumably shorewall knows which these are when it starts up, 
so creating a list of files to check would be easy.

 > b) Extension scripts -- there can be an extension script for any 
chain > in the old configuration. So if you change one of them, you would
 > expect that "shorewall restart" would reprocess all of the
config
 > files, right? That means that you have to actually KNOW the names of
 > the files that you need to check; they aren''t just limited to the
 > released Shoreall configuration files.

Yes. I assume that, when it starts normally, shorewall would save a list 
of these files. Then, the next time it starts, it would quickly check 
the modification times of these, and if none have been changed, it can 
use the cache.


 > c) I also worry about people starting a configuration that
doesn''t
 > work and having Shorewall stupidly save that configuration and then
 > continue to slavishly reinstate it every time the user attempts to
 > correct the situation by typing "shorewall restart" or (as most
naive
 > users do) rebooting. It''s important that the state _only be saved
when
 > it is a valid state_.

I agree. But my suggestion doesn''t affect this. If the user types 
"shorewall restart", then they will get a state that corresponds to
the
most recent rules in /etc/shorewall. If the user has changed these rules 
to fix a broken configuration, then my cache would be discarded. If the 
user hasn''t changed them, they will get a broken result, whether or not
the cache is used.


 > d) Unfortunately, this isn''t easy. I''ve thought of
trying to do
 > something like this but I really don''t want to spend the cycles
it
 >would require to do it right just so that people don''t have to
type
 >"shorewall save" at a root shell prompt. And the user KNOWS when
the
 > configuration is right or wrong -- Shorewall doesn''t.


I think it may actually be easier than you suggest. My suggestion, given 
in the previous email was slightly complicated by the need to have 3 
behaviours rather than 2. However, if we ignore the precedence of a 
manual "shorewall save" for now, I think the rules boil down to this:


1)Shorewall starts up normally. As it does so, it saves a list of all 
the files it parses, with their modification times. (filemtimes.last) If 
it succeeds in starting, it saves its state. (shorewall.cache)

2)The next time, shorewall starts, [X] it first reads filemtimes.last.
It then checks all of the files referenced, to see if they have changed.
If they have not, then it quickly restores the firewall from 
shorewall.cache. Otherwise, go to (1).


To cope with the existence of the manually created
/var/lib/shorewall/restore, it should be tested for at [X].



Re. startup times, shorewall takes me 4 seconds to start on a P4 3.2 
GHz. The 15 seconds measured was on an old (450 MHz) laptop, but in this 
case, it really would be useful to have the speed.


Just to check: If one saves a known shorewall configuration (shorewall 
save), and then the IP addresses of the interfaces change (eg DHCP), is 
the restored state valid?


Best wishes

Richard


P.S. Sorry for not quoting the reply properly. I tried, but Mozilla 
assumed that everything in the original message below "--" was a 
signature, and cut it all out! So I''m forwarding rather than replying.