Hi, it seems not to be possible to add more than one host at once to a zone. So shorewall add br0:eth0:192.168.2.10,eth0:192.168.2.11 work fails, since "br0:eth0:192.168.2.10,eth0" is interpreted as one interface. --snip -- iptables v1.2.9: interface name `eth0:192.168.2.10,eth0'' must be shorter than IFNAMSIZ (15) Try `iptables -h'' or ''iptables --help'' for more information. Error: Can''t add -A to zone br0_dynf -- snip -- However, nevertheless the entry is added somewhere (but with no effect at all), since ''show zones'' throws work br0:eth0:192.168.2.10,eth0:192.168.2.11 Would it be possible to implement the same syntax possibilities in ''shorewall add'' and ''delete'' as in the /etc/shorewall/hosts file? The reason is that I wrote a web interface which allows adminsitrators to give access rights to clients in a computer pool. I tried to dynamically add about 40 hosts to 2 overlapping zones (one for printing, one for www). This took the program more than 3 minutes, adding one by one. ''top'' showed that most of the time is consumed by the shorewall shell program, not by iptables itself. I think the reason is that on every call of the ''add'' command shorewall loads all the modules etc. Thus, adding the same parsing for the command line as for the hosts file should speed it up a bit. I was looking through the code of /usr/share/shorewall/firewall to find the piece where shorewall is interpreting the hosts file and to copy this code to the ''shorewall add'' functionality, but I didn''t find it. But, I''m quite inexpierenced in shell programming. However, a short hint where to find the hosts parsing would help me to customize this. Thanks a lot /ben
On Sun, 2004-12-05 at 23:09 +0100, Ben Greiner wrote:> However, a short hint where to find the hosts parsing would help me to > customize this.I doubt it. The add/delete code is the most fragile part of Shorewall. It is very easy to break is subtle ways. Parsing a list as opposed to single item is trivial (although for a command line, I would use white space as the separator rather than comma). At any rate, the hosts file is parsed in validate_hosts_file(). -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
On Sun, 2004-12-05 at 15:21 -0800, Tom Eastep wrote:> On Sun, 2004-12-05 at 23:09 +0100, Ben Greiner wrote: > > > However, a short hint where to find the hosts parsing would help me to > > customize this. > > I doubt it. The add/delete code is the most fragile part of Shorewall. > It is very easy to break is subtle ways. Parsing a list as opposed to > single item is trivial (although for a command line, I would use white > space as the separator rather than comma). > > At any rate, the hosts file is parsed in validate_hosts_file().Attached is a completely untested patch to accept a space-separated list of addresses: add <interface>:[<bridge port>][:<address>] ... <zone> The delete command is similar. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
On Sun, 2004-12-05 at 15:33 -0800, Tom Eastep wrote:> On Sun, 2004-12-05 at 15:21 -0800, Tom Eastep wrote: > > On Sun, 2004-12-05 at 23:09 +0100, Ben Greiner wrote: > > > > > However, a short hint where to find the hosts parsing would help me to > > > customize this. > > > > I doubt it. The add/delete code is the most fragile part of Shorewall. > > It is very easy to break is subtle ways. Parsing a list as opposed to > > single item is trivial (although for a command line, I would use white > > space as the separator rather than comma). > > > > At any rate, the hosts file is parsed in validate_hosts_file(). > > Attached is a completely untested patch to accept a space-separated list > of addresses: > > add <interface>:[<bridge port>][:<address>] ... <zone> > > The delete command is similar.I believe that there are serious problems with the patch -- there are steps that need to factored outside of the loop. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
On 06.12.2004 00:41, Tom Eastep wrote:>I believe that there are serious problems with the patch -- there are >steps that need to factored outside of the loop. > >-Tom > >Hi Tom, just saw your quick answer. I tried it differently: by changing the add_to_zone function itself. I just had to change about 10 lines, somewhere in the beginning and at the very end. As far I can read from the shorewall status output, it seems to work accordingly. Syntax to call is exactly like /etc/shorewall/rules, i.e. comma-separated. However, so far I did not work on ''delete'', error messages might still throw the whole $1, and since startup_error immediately stops execution the function will stop the loop at the point it finds an error. Overall it''s not very elegant, but ... Below the changes. My additions are marked with comments and "BG". Version is Shorewall-2.2.0-Beta7. add_to_zone() # $1 = <interface>[:<hosts>] $2 = zone { -- snip -- # # Isolate interface and host parts # interface=${1%%:*} # BG: now take it as a list of hosts named nhosts, old line: host=${1#*:} nhosts=${1#*:} # BG: need a var to implement that validate_interfaces_file is run only once validate_files="yes" # BG: now do the following stuff for all hosts in nhosts... for host in $(separate_list $nhosts); do [ -z "$host" ] && host="0.0.0.0/0" # # Load $zones # determine_zones # # Validate Interfaces File # # BG: Validate only for first loop (because of error: Duplicate interface when run twice (?)) if [ "$validate_files" = "yes" ]; then validate_interfaces_file validate_files="no" fi # # Validate IPSec File # -- snip --- ... -- snip --- done < ${STATEDIR}/chains # BG: moved down: rm -rf $TMP_DIR # BG: change message to $newhost progress_message "$newhost added to zone $2" # BG: end for all hosts done # BG: moved here: rm -rf $TMP_DIR } I attach the complete diff file. What do you think? Cheers /ben
On 06.12.2004 01:48, Ben Greiner wrote:> > > What do you think? > > Cheers > > /benI just implemented the same stuff in delete_from_zone(). Works nice. Then I compared shorewall status between a) adding one by one b) using the patch (adding all at once) and a) deleting one by one b) using the patch (deleting all at once) I found no differences. However, for both styles, when adding and then deleting dynamic hosts, rules concerning these are not deleted fully. The rules in chain br0_dynf _between_ these dynamic hosts (and only these rules) are not deleted. Actually, I don''t know what that means. Cheers /ben
Here some benchmarks: Adding and then deleting 34 hosts to 1 zone, on double Xeon 2.8, 4GB, SCSI Raid 1 Adding with /bin/sh program one by one real 0m37.853s user 0m10.273s sys 0m28.297s Deleting with /bin/sh program one by one real 0m37.737s user 0m10.470s sys 0m27.882s Adding with patch real 0m24.381s user 0m5.177s sys 0m19.509s Deleting with patch real 0m23.624s user 0m5.436s sys 0m18.470s And a simple shorewall restart real 0m4.652s user 0m1.498s sys 0m3.188s
On Mon, 2004-12-06 at 03:33 +0100, Ben Greiner wrote:> Here some benchmarks: > > Adding and then deleting 34 hosts to 1 zone, on double Xeon 2.8, 4GB, > SCSI Raid 1 > > Adding with /bin/sh program one by one > real 0m37.853s > user 0m10.273s > sys 0m28.297s > > Deleting with /bin/sh program one by one > real 0m37.737s > user 0m10.470s > sys 0m27.882s > > Adding with patch > real 0m24.381s > user 0m5.177s > sys 0m19.509s > > Deleting with patch > real 0m23.624s > user 0m5.436s > sys 0m18.470s > > And a simple > shorewall restart > real 0m4.652s > user 0m1.498s > sys 0m3.188s >You should understand that "add" and "delete" was created to be able to handle Roadwarrior logons and logoffs; it was never intended for mass changes to zone membership... -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
On Sun, 2004-12-05 at 19:33 -0800, Tom Eastep wrote:> > You should understand that "add" and "delete" was created to be able to > handle Roadwarrior logons and logoffs; it was never intended for mass > changes to zone membership...Wow -- I''ve got to start reading this stuff before I send it. s/b "You should understand that "add" and "delete" *were* created..." -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
On Mon, 2004-12-06 at 02:12 +0100, Ben Greiner wrote:> On 06.12.2004 01:48, Ben Greiner wrote: > > > > > > > What do you think?I need some time to look at it. I won''t include any change like this in 2.2.0 because it is too easy to mess up this code (as you report below, it is already wrong :-( ).> > > > Cheers > > > > /ben > > I just implemented the same stuff in delete_from_zone(). Works nice. > > Then I compared shorewall status between > a) adding one by one b) using the patch (adding all at once) > > and > > a) deleting one by one b) using the patch (deleting all at once) > > I found no differences. > > However, for both styles, when adding and then deleting dynamic hosts, > rules concerning these are not deleted fully. > The rules in chain br0_dynf _between_ these dynamic hosts (and only > these rules) are not deleted. Actually, I don''t know what that means.It means that for some reason, the -D command at line 6943 doesn''t match the -A command at line 6796. If you trace the "add" and the "delete" commands, you should be able to see the difference. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
On Mon, 2004-12-06 at 07:22 -0800, Tom Eastep wrote:> On Mon, 2004-12-06 at 02:12 +0100, Ben Greiner wrote: > > On 06.12.2004 01:48, Ben Greiner wrote: > > > > > > > > > > > What do you think? > > I need some time to look at it. I won''t include any change like this in > 2.2.0 because it is too easy to mess up this code (as you report below, > it is already wrong :-( ). > > > > > > > Cheers > > > > > > /ben > > > > I just implemented the same stuff in delete_from_zone(). Works nice. > > > > Then I compared shorewall status between > > a) adding one by one b) using the patch (adding all at once) > > > > and > > > > a) deleting one by one b) using the patch (deleting all at once) > > > > I found no differences. > > > > However, for both styles, when adding and then deleting dynamic hosts, > > rules concerning these are not deleted fully. > > The rules in chain br0_dynf _between_ these dynamic hosts (and only > > these rules) are not deleted. Actually, I don''t know what that means. > > It means that for some reason, the -D command at line 6943 doesn''t match > the -A command at line 6796. > > If you trace the "add" and the "delete" commands, you should be able to > see the difference.FWIW, I can''t reproduce this bug: ursa:/etc/test # shorewall add eth1:1.2.3.4 loc Loading /usr/share/shorewall/functions... Processing /etc/shorewall/params ... Processing /etc/shorewall/shorewall.conf... Processing /etc/shorewall/ipsec... eth1:1.2.3.4 added to zone loc ursa:/etc/test # shorewall show eth1_dynf Shorewall-2.2.0-Beta6 Chain eth1_dynf at ursa - Mon Dec 6 07:29:15 PST 2004 Counters reset Fri Dec 3 13:35:04 PST 2004 Chain eth1_dynf (1 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT all -- * eth0 1.2.3.4 192.168.1.0/24 policy match dir out pol none 0 0 all2all all -- * eth1 1.2.3.4 0.0.0.0/0 policy match dir out pol none 0 0 loc2sec all -- * eth1 1.2.3.4 0.0.0.0/0 policy match dir out pol none 0 0 all2all all -- * eth1 0.0.0.0/0 1.2.3.4 policy match dir out pol none 0 0 sec2loc all -- * eth1 0.0.0.0/0 1.2.3.4 policy match dir out pol none ursa:/etc/test # shorewall delete eth1:1.2.3.4 loc Loading /usr/share/shorewall/functions... Processing /etc/shorewall/params ... Processing /etc/shorewall/shorewall.conf... Processing /etc/shorewall/ipsec... eth1:1.2.3.4 removed from zone loc ursa:/etc/test # shorewall show eth1_dynf Shorewall-2.2.0-Beta6 Chain eth1_dynf at ursa - Mon Dec 6 07:29:31 PST 2004 Counters reset Fri Dec 3 13:35:04 PST 2004 Chain eth1_dynf (1 references) pkts bytes target prot opt in out source destination ursa:/etc/test # -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
On 06.12.2004 16:31, Tom Eastep wrote:> FWIW, I can''t reproduce this bug: > >ursa:/etc/test # shorewall add eth1:1.2.3.4 loc >Loading /usr/share/shorewall/functions... >Processing /etc/shorewall/params ... >Processing /etc/shorewall/shorewall.conf... >Processing /etc/shorewall/ipsec... >eth1:1.2.3.4 added to zone loc > >>ursa:/etc/test # shorewall delete eth1:1.2.3.4 loc >Loading /usr/share/shorewall/functions... >Processing /etc/shorewall/params ... >Processing /etc/shorewall/shorewall.conf... >Processing /etc/shorewall/ipsec... >eth1:1.2.3.4 removed from zone loc >ursa:/etc/test # shorewall show eth1_dynf >Shorewall-2.2.0-Beta6 Chain eth1_dynf at ursa - Mon Dec 6 07:29:31 PST >2004 > >Counters reset Fri Dec 3 13:35:04 PST 2004 > >Chain eth1_dynf (1 references) > pkts bytes target prot opt in out source >destination >ursa:/etc/test # > >-Tom > >It does not remove the chain entries _between_ dynamically added hosts. Try with two hosts. /ben -- _____________________________________________________________________________ Ben Greiner Universität zu Köln/University of Cologne Staatswissenschaftliches Seminar Lehrstuhl Prof. Dr. Ockenfels Albertus-Magnus-Platz 50923 KÖLN, GERMANY PHONE ++49 (0) 221 470 6116 E-MAIL bgreiner@uni-koeln.de http://ockenfels.uni-koeln.de
On Mon, 2004-12-06 at 19:20 +0100, Ben Greiner wrote:> > > It does not remove the chain entries _between_ dynamically added hosts. > Try with two hosts.Still no luck -- please send me an "add" trace and a "delete" trace where the "delete" doesn''t remove a rule added by the "add". -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
On Tue, 2004-12-07 at 15:16 -0800, Tom Eastep wrote:> On Mon, 2004-12-06 at 19:20 +0100, Ben Greiner wrote: > > > > > > It does not remove the chain entries _between_ dynamically added hosts. > > Try with two hosts. > > Still no luck -- please send me an "add" trace and a "delete" trace > where the "delete" doesn''t remove a rule added by the "add". >I discovered some problems with add/delete and ipsec so while I was fixing those, I went ahead and added the ability to add/delete multiple hosts in one command. The code is in CVS (Shorewall2/) -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key