I have a dual ISP shorewall[-lite] (4.0.5) installation. It happens
most times on boot that one of the ISP interfaces will not be quite
fully plumbed before shorewall-lite wants to restore the firewall.
So my network interfaces start to come up but before the PPPoE interface
is fully up, shorewall-lite starts it''s "restore" (i.e. on
boot) process
and inevitably I get an error:
ERROR: Can''t determine the IP address of ppp0
and Shorewall is not started.
Now, if I only had the one ISP interface I wouldn''t care so much
because
Shorewall doesn''t do me much good without an Internet connection
anyway.
But that is not the case. It''s a perfectly valid situation that one of
my connections is up and I would want Shorewall to start and function in
that case.
It would appear this error stems from a call to
find_first_interface_address() in define_firewall():
define_firewall() {
clear_routing_and_traffic_shaping
#
# Establish the values of shell variables used in the following function
calls
#
PPP0_ADDRESS=$(find_first_interface_address ppp0)
ETH0_1_ADDRESS=$(find_first_interface_address eth0.1)
I''m not sure if there is anything specific in my configuration which
is driving this need to determine the interface addresses although it
appears to be MultiISP related as setup_routing_and_traffic_shaping()
want to do:
gateway=$(detect_gateway ppp0)
if [ -n "$gateway" ]; then
run_ip route replace $PPP0_ADDRESS dev ppp0 table 2
run_ip route add default via $gateway dev ppp0 table 2
else
fatal_error "Unable to detect the gateway through interface ppp0"
fi
What''s interesting is that the block of code above, that wants to use
$PPP0_ADDRESS is guarded by an "interface_is_usable ppp0" test and if
it''s not usable:
fatal_error "Interface ppp0 is not configured -- Provider IGS (2)
Cannot be Added"
I guess what I''m wondering is if there is a way to make this more
robust
such that the absence of an interface just means not doing the
configuration bits for it rather than being fatal.
Presumably I would restart shorewall-lite any time a new interface was
plumbed to take into consideration the additional information.
What would be "the cat''s ass" mind you, would be for
shorewall to
produce little scriptlets that added and removed appropriate
configuration items when an individual interface appeared or disappeared
and those could be plugged into the "hotplug" infrastructure, making
the
entire restart of shorewall unnecessary and overkill. But I think that
is a much more difficult (albeit interesting) task that what I''m
proposing/requesting above.
Thots?
b.
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Brian J. Murrell wrote:> I''m not sure if there is anything specific in my configuration which > is driving this need to determine the interface addresses although it > appears to be MultiISP related as setup_routing_and_traffic_shaping() > want to do: > > gateway=$(detect_gateway ppp0) > > if [ -n "$gateway" ]; then > run_ip route replace $PPP0_ADDRESS dev ppp0 table 2 > run_ip route add default via $gateway dev ppp0 table 2 > else > fatal_error "Unable to detect the gateway through interface ppp0" > fi > > What''s interesting is that the block of code above, that wants to use > $PPP0_ADDRESS is guarded by an "interface_is_usable ppp0" test and if > it''s not usable: > > fatal_error "Interface ppp0 is not configured -- Provider IGS (2) Cannot be Added" > > I guess what I''m wondering is if there is a way to make this more robust > such that the absence of an interface just means not doing the > configuration bits for it rather than being fatal.Brian, If you would have spent half the time reading the output of ''man shorewall-providers'' as you did reading the generated code, you would have found the ''optional'' provider option. It does exactly what you are asking for. -tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Wed, Jan 30, 2008 at 11:22:47PM -0500, Brian J. Murrell wrote:> What would be "the cat''s ass" mind you, would be for shorewall to > produce little scriptlets that added and removed appropriate > configuration items when an individual interface appeared or disappeared > and those could be plugged into the "hotplug" infrastructure, making the > entire restart of shorewall unnecessary and overkill.This would be an inadequate solution, because it does not account for ethernet interfaces that change media state; it pretty much only works for pppd. Hotplug is not the right approach. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Thu, 2008-01-31 at 13:25 +0000, Andrew Suffield wrote:> > This would be an inadequate solution, because it does not account for > ethernet interfaces that change media state;As in link up/down -- the result of [un]plugging the cable? I don''t think I''m following. Can you expand? b. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Thu, Jan 31, 2008 at 08:44:14AM -0500, Brian J. Murrell wrote:> On Thu, 2008-01-31 at 13:25 +0000, Andrew Suffield wrote: > > > > This would be an inadequate solution, because it does not account for > > ethernet interfaces that change media state; > > As in link up/down -- the result of [un]plugging the cable?Yes, but also of the device on the other end changing state. For example, it''s fairly common for a fibre transceiver to indicate loss of link on the fibre connection by switching off its ethernet interface. It''s also the behaviour you get when somebody puts a nail through the cat5 cable. From userspace, this appears as the RUNNING flag changing on the interface, not as a hotplug event. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Thu, 2008-01-31 at 13:51 +0000, Andrew Suffield wrote:> On Thu, Jan 31, 2008 at 08:44:14AM -0500, Brian J. Murrell wrote: > > On Thu, 2008-01-31 at 13:25 +0000, Andrew Suffield wrote: > > > > > > This would be an inadequate solution, because it does not account for > > > ethernet interfaces that change media state; > > > > As in link up/down -- the result of [un]plugging the cable? > > Yes, but also of the device on the other end changing state. For > example, it''s fairly common for a fibre transceiver to indicate loss > of link on the fibre connection by switching off its ethernet > interface. It''s also the behaviour you get when somebody puts a nail > through the cat5 cable. From userspace, this appears as the RUNNING > flag changing on the interface, not as a hotplug event.Hrm. Maybe I inadvertently described my goal more lofty than it is. I guess the "event" that I want to plug into is more like when an administrator (or a system boot) if{up,down}''s an interface. I don''t know how many distributions provide hooks into that, but OpenWrt for one does. b. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Wed, 2008-01-30 at 22:19 -0800, Tom Eastep wrote:> > Brian, > > If you would have spent half the time reading the output of ''man > shorewall-providers'' as you did reading the generated code, you would > have found the ''optional'' provider option. It does exactly what you are > asking for.Doh! Always assuming the solution is high-tech. I will give that a shot. I still think fragments of configuration that can be applied/de-applied on interface addition/removal is an interesting idea. Oh, but to have the time... ~sigh~ Thanx Tom! b. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Brian J. Murrell wrote:>I still think fragments of configuration that can be applied/de-applied >on interface addition/removal is an interesting idea. Oh, but to have >the time... ~sigh~You can always trigger a shorewall reload. So taking Debian as an example, in your /etc/networks/interfaces file you could do something like : iface eth0 inet static ... up shorewall restart down shorewall restart (I think that''s the syntax). Then whenever you take the interface up or down you will automatically restart shorewall. I believe shorewall has locking so it won''t break anything if two different processes both call for a restart. Particularly now we have the Perl version, reloads are quite quick so it''s hardly any problem to reload the whole thing. By way of comparison, I''ve ported an accounting box at work to the newer version. As well as traffic shaping, it does accounting for in and out traffic on an entire class C - so 510 accounting rules or so. Hardware is Pentium III 1GHz and whilst the older version took about 90 seconds to load, the newer Perl version loads it in about 6 seconds. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Thu, 2008-01-31 at 20:25 +0000, Simon Hobson wrote:> Brian J. Murrell wrote: > > >I still think fragments of configuration that can be applied/de-applied > >on interface addition/removal is an interesting idea. Oh, but to have > >the time... ~sigh~ > > You can always trigger a shorewall reload. So taking Debian as an > example, in your /etc/networks/interfaces file you could do something > like : > > iface eth0 inet static > ... > up shorewall restart > down shorewall restartYes, indeed, that''s exactly the kind of hook where I''d put a fragment script fragment generated by shorewall to do the necessary bits to add/remove that interface to the running configuration.> (I think that''s the syntax). Then whenever you take the interface up > or down you will automatically restart shorewall.Right. I''m just proposing taking a more surgical approach.> I believe shorewall > has locking so it won''t break anything if two different processes > both call for a restart.That''s good to know.> Particularly now we have the Perl version, reloads are quite quick so > it''s hardly any problem to reload the whole thing.In my case, my target is a shorewall-lite machine, but yes, a "shorewall restore" is still quite quick at loading. My proposal is simply to modify the running configuration to just adjust for the interface change so that there is less disruption.> By way of comparison, I''ve ported an accounting box at work to the > newer version. As well as traffic shaping, it does accounting for in > and out traffic on an entire class C - so 510 accounting rules or so. > Hardware is Pentium III 1GHz and whilst the older version took about > 90 seconds to load, the newer Perl version loads it in about 6 > seconds.Right. But there is a scaling issue here. As the number of rules grows and the number of interface changes grows so does the periodic outages due to entire ruleset/routing/traffic control reloads. b. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Tue, Feb 12, 2008 at 08:27:52AM -0500, Brian J. Murrell wrote:> > By way of comparison, I''ve ported an accounting box at work to the > > newer version. As well as traffic shaping, it does accounting for in > > and out traffic on an entire class C - so 510 accounting rules or so. > > Hardware is Pentium III 1GHz and whilst the older version took about > > 90 seconds to load, the newer Perl version loads it in about 6 > > seconds. > > Right. But there is a scaling issue here. As the number of rules grows > and the number of interface changes grows so does the periodic outages > due to entire ruleset/routing/traffic control reloads.It would be nice if the outage could be completely eliminated. However, this is a problem for the kernel people - we''d need atomic whole-configuration changes in netfilter and tc, rather than the current rule-at-a-time system. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Tue, 2008-02-12 at 13:37 +0000, Andrew Suffield wrote:> > It would be nice if the outage could be completely > eliminated. However, this is a problem for the kernel people - we''d > need atomic whole-configuration changes in netfilter and tc, rather > than the current rule-at-a-time system.Indeed, but we could take quite a large step towards that goal by only modifying an existing configuration to make the changes needed to affect an interface change. Anyway, unless this topic takes an interesting turn in dicussion, I will stop beating this drum. I''ve put the idea out there for consideration but unfortunately don''t really have the time to hack on it myself. My perl-fu is getting older and rustier as each day goes by as well. Cheers, b. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Brian J. Murrell wrote:> > It would be nice if the outage could be completely >> eliminated. However, this is a problem for the kernel people - we''d >> need atomic whole-configuration changes in netfilter and tc, rather >> than the current rule-at-a-time system. > >Indeed, but we could take quite a large step towards that goal by only >modifying an existing configuration to make the changes needed to affect >an interface change.The problem there is that it''s a shedload more complicated to work out what needs to be changed than it is to build it in the first place. I don''t suppose there''s provision for renaming of chains etc in situ ? That would appear (at first sight anyway) to be the easiest compromise - build a new set of rules, make them active by changing names, remove the old ones. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Brian J. Murrell wrote:> On Tue, 2008-02-12 at 13:37 +0000, Andrew Suffield wrote: >> It would be nice if the outage could be completely >> eliminated. However, this is a problem for the kernel people - we''d >> need atomic whole-configuration changes in netfilter and tc, rather >> than the current rule-at-a-time system.We do have that now with Netfilter under Shorewall-perl (one atomic update per table). But nothing for the other bits.> > Indeed, but we could take quite a large step towards that goal by only > modifying an existing configuration to make the changes needed to affect > an interface change. > > Anyway, unless this topic takes an interesting turn in dicussion, I will > stop beating this drum. I''ve put the idea out there for consideration > but unfortunately don''t really have the time to hack on it myself. My > perl-fu is getting older and rustier as each day goes by as well.IIRC, this thread started with the lament that your PPP device was taking so long to come up that Shorewall was getting started ahead of it. If that is the immediate problem that you are trying to solve, there is a capability in the Debian init script (init.debian.sh) that you should take advantage of in your OpenWRT package. The script defines a ''wait_interface'' variable which may be set in /etc/default/shorewall and contains a space-separated list of interface names. The ''start'' command waits for 90 seconds for each interface to come up (you can hack the script to change the timeout). While the standard init script doesn''t include support for ''wait-interface'', the ''waitforifup'' script (which does the actual waiting) is installed in /usr/share/shorewall and can be invoked from the ''init'' extension script. As to the loftier goal of dynamically reacting to interfaces going up and down like yo-yos, I''ve long resisted producing any Shorewall code that dynamically modifies the configuration in reaction to changes in the environment. Among the reasons are: 1) The supportability of the product suffers. A static dump of the running configuration is no longer sufficient to analyze problems. The original configuration and a detailed log of alterations are also required. 2) A whole new class of problem is created -- "My firewall worked correctly for 27 days and then it suddenly quit working -- what''s wrong?". Today, we can be sure that such problems are not Shorewall-related and I like it that way. 3) The ''save'' and ''restore'' commands become nearly impossible to implement so that they behave correctly (they work questionably today when DYNAMIC_ZONES=Yes). -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Tue, Feb 12, 2008 at 07:56:20AM -0800, Tom Eastep wrote:> Brian J. Murrell wrote: >> On Tue, 2008-02-12 at 13:37 +0000, Andrew Suffield wrote: >>> It would be nice if the outage could be completely >>> eliminated. However, this is a problem for the kernel people - we''d >>> need atomic whole-configuration changes in netfilter and tc, rather >>> than the current rule-at-a-time system. > > We do have that now with Netfilter under Shorewall-perl (one atomic > update per table).I don''t believe so - my understanding of iptables-restore (based on studying its code) is that from the perspective of the kernel it is equivalent to running all the iptables commands by hand, so it''s not atomic, it just doesn''t have to reload the iptables binary and libraries a few hundred times. But it''s possible that I''m missing something. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Tue, 2008-02-12 at 07:56 -0800, Tom Eastep wrote:> > IIRC, this thread started with the lament that your PPP device was > taking so long to come up that Shorewall was getting started ahead of > it.Yeah. That is just one of a number of issues with multiple interfaces and the fact that one could be up and the other not though. Generally for people with a single interface, if it''s not up, who cares what state shorewall and routing are in.> If that is the immediate problem that you are trying to solve, there > is a capability in the Debian init script (init.debian.sh) that you > should take advantage of in your OpenWRT package. The script defines a > ''wait_interface'' variable which may be set in /etc/default/shorewall and > contains a space-separated list of interface names. The ''start'' command > waits for 90 seconds for each interface to come up (you can hack the > script to change the timeout).OpenWRT != Debian. It handles interfaces in a completely different manner.> As to the loftier goal of dynamically reacting to interfaces going up > and down like yo-yos, I''ve long resisted producing any Shorewall code > that dynamically modifies the configuration in reaction to changes in > the environment. Among the reasons are: > > 1) The supportability of the product suffers. A static dump of the > running configuration is no longer sufficient to analyze problems. The > original configuration and a detailed log of alterations are also required. > > 2) A whole new class of problem is created -- "My firewall worked > correctly for 27 days and then it suddenly quit working -- what''s > wrong?". Today, we can be sure that such problems are not > Shorewall-related and I like it that way. > > 3) The ''save'' and ''restore'' commands become nearly impossible to > implement so that they behave correctly (they work questionably today > when DYNAMIC_ZONES=Yes).Those are all valid points which I won''t argue with. The only thing I''d say for the last point is that the process of restore could be: - restore interface agnostic bits - for each interface_up; do restore interface specifics done The other major pain with multiple interfaces and Shorewall is the handling of default routes. When an interface goes down, typically the O/S support wants to remove a default route. With Shorewall''s use of multiple default routes and routing tables and ip rules, this almost always never works. Same situation for when the interface comes back up. Sadly 99% of O/S support assumes that there is only one interface and "route add default bla bla bla" is all that is needed. One could opt to not have interface plumbing scripts define default routes (where that option is even possible) and have Shorewall just do it with the restore option on every interface change, but again that''s a heavier operation than I''m proposing. But even the "O/S doesn''t plumb default routes and leaves it to Shorewall restore to do" option does not always work. If a broadcast type interface (i.e. cable modem) does not plumb a default route when it comes up, Shorewall has no idea what the default router is and emits a "I don''t know what the default route for interface bla is" type message. So it''s kinda damned if you do, damned if you don''t. Sure, it''s possible to find a solution for this wart, probably, but I guess I was just thinking of the bigger picture. b. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Brian J. Murrell wrote:> OpenWRT != Debian. It handles interfaces in a completely different > manner.The feature is implemented in a completely distribution-neutral way (it uses the same logic to determine when an interface is up as is used in testing if an optional interface is usable).> > Those are all valid points which I won''t argue with. The only thing I''d > say for the last point is that the process of restore could be: > > - restore interface agnostic bits > - for each interface_up; do > restore interface specifics > doneAnd how do you tell iptables-save/iptables-restore what the interface-agnostic bits are? You can''t, so you end up having to write your own iptables-save -- in Bourne shell.> > The other major pain with multiple interfaces and Shorewall is the > handling of default routes.<rest of rant omitted> The issue here is that the Shorewall Multi-ISP feature is a hack to work around the fact that many Shorewall users are cheap and try to use a pair of consumer-grade uplinks (often with dynamic IP addresses) to effect a fault-tolerant solution. The problem of maintaining accurate routing tables in the face of changing network topology is effectively solved through the use of interior gateway routing protocols but the consumer-grade services employed by most Shorewall users don''t offer support for such protocols. -Tom (who must get to his real job now) -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Simon Hobson wrote:> > I don''t suppose there''s provision for renaming of chains etc in situ > ? That would appear (at first sight anyway) to be the easiest > compromise - build a new set of rules, make them active by changing > names, remove the old ones.iptables does support such a capability. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Tue, 2008-02-12 at 08:40 -0800, Tom Eastep wrote:> Brian J. Murrell wrote: > > The feature is implemented in a completely distribution-neutral way (it uses > the same logic to determine when an interface is up as is used in testing if > an optional interface is usable).Ahh. Yes. Sorry for misreading. I thought you were referring to a feature in /etc/network/interfaces on Debian machines. I could take advantage of that. The pros/cons of waiting vs. just reloading twice within what could wind up being a relative short period of time would be an interesting small study.> And how do you tell iptables-save/iptables-restore what the > interface-agnostic bits are? You can''t, so you end up having to write your > own iptables-save -- in Bourne shell.Ahh. Yes. I see what you mean. I guess I was thinking there are 2 or more different bits. The bulk of the rules which are agnostic to what (Internet) interfaces are up and down and those could be loaded with iptables-restore. Then there are the particular bits that need to wait for interfaces and I would propose just adding/changing/removing those on interface change with some iptables rules.> The issue here is that the Shorewall Multi-ISP feature is a hack to work > around the fact that many Shorewall users are cheap"Consumers", yes.> and try to use a pair of > consumer-grade uplinks (often with dynamic IP addresses) to effect a > fault-tolerant solution.Indeed.> The problem of maintaining accurate routing tables > in the face of changing network topology is effectively solved through the > use of interior gateway routing protocols but the consumer-grade services > employed by most Shorewall users don''t offer support for such protocols.Indeed, absolutely agreed.> -Tom (who must get to his real job now)NP. As I said, I''ve floated my idea and really not wanting to beat a dead horse. Apart from that one ugly wart in that a broadcast domain/DHCP service has to be allowed to populate a default route or else shorewall barfs because it can''t find the default route -- it mostly works. I have not thought terribly much about how to solve that yet. Ideally, the interface plumbing doesn''t touch the routing tables and leaves shorewall to do it. b. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Tue, 2008-02-12 at 09:02 -0800, Tom Eastep wrote:> iptables does support such a capability.No, but I think the same could be effected by uniquely naming chains so that two of the same chain can be installed at the same time (i.e. current and new), and then just changing a redirection from the current chain to the new one. One could then lazily clean up what was current (and is now old). Yes? I''m not sure how "elegant" that is, but it seems doable. b. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Brian J. Murrell wrote:> On Tue, 2008-02-12 at 09:02 -0800, Tom Eastep wrote: >> iptables does support such a capability. > > NoYes -- the -E command renames an existing chain. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Tom Eastep wrote:> Brian J. Murrell wrote: >> On Tue, 2008-02-12 at 09:02 -0800, Tom Eastep wrote: >>> iptables does support such a capability. >> No > > Yes -- the -E command renames an existing chain. >Example: ursa:~ # iptables -N foo ursa:~ # iptables -N bar ursa:~ # iptables -A foo -j bar ursa:~ # iptables -L foo Chain foo (0 references) target prot opt source destination bar all -- anywhere anywhere ursa:~ # iptables -E bar baz ursa:~ # iptables -L foo Chain foo (0 references) target prot opt source destination baz all -- anywhere anywhere ursa:~ # -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Tue, Feb 12, 2008 at 11:15:55AM -0500, Brian J. Murrell wrote:> The other major pain with multiple interfaces and Shorewall is the > handling of default routes.Shorewall is a terrible solution to routing problems, it''s just got a handful of tricks that work for a few relatively common cases. If you''re doing anything complex with routing, you should be doing it with something else (like quagga), and leave shorewall to the things it''s good at. It is much easier to run shorewall alongside a real routing daemon than it is to try doing all that stuff with just shorewall. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Tue, Feb 12, 2008 at 09:46:20AM -0800, Tom Eastep wrote:> ursa:~ # iptables -L foo > Chain foo (0 references) > target prot opt source destination > bar all -- anywhere anywhere > ursa:~ # iptables -E bar baz > ursa:~ # iptables -L foo > Chain foo (0 references) > target prot opt source destination > baz all -- anywhere anywhere > ursa:~ #Which is why it''s not useful here - it has no effect on the structure, it just changes the name, so it cannot be used as an atomic-insert-of-many-rules. The necessary feature would be a ''swap chains'' command, that replaced all references to one existing chain with references to another existing chain (not that I''m advocating it). ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Tue, 2008-02-12 at 18:30 +0000, Andrew Suffield wrote:> > Shorewall is a terrible solution to routing problems, it''s just got a > handful of tricks that work for a few relatively common cases.Agreed. I don''t know that there is a better solution for the MultiISP problem though. The tricks shorewall has to play with provider routing tables and routing rules, etc.> If > you''re doing anything complex with routing, you should be doing it > with something else (like quagga), and leave shorewall to the things > it''s good at.That''s a fair point, except that I don''t know that quagga actually does anything to solve the problem. The problem being specifically, the management of multiple default routes on interface availability. AFAIK, quagga will not plumb default routes via a newly available interface and replumb routes when an interface goes away. Quagga will also not manage the required provider tables to ensure the symmetric routing that is required.> It is much easier to run shorewall alongside a real routing daemon > than it is to try doing all that stuff with just shorewall.I do run quagga on my gateway and would love it if quagga solved the problems, but AFAICT, it does not. :-( b. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Tue, Feb 12, 2008 at 01:42:19PM -0500, Brian J. Murrell wrote:> That''s a fair point, except that I don''t know that quagga actually does > anything to solve the problem. The problem being specifically, the > management of multiple default routes on interface availability. AFAIK, > quagga will not plumb default routes via a newly available interface and > replumb routes when an interface goes away.If you place ''static'' routes in zebra.conf, thusly: ip route 10.1.0.1/32 10.1.100.2 then quagga will insert and remove them as their destinations become available and go away, for whatever reason, including link status. This does work with default routes as well. That combined with suitably crafted routing tables should be sufficient, if I''ve understood your problem correctly.> Quagga will also not manage > the required provider tables to ensure the symmetric routing that is > required.Indeed, you have to do that bit on your own - but it''s static and quite simple. I haven''t actually built one of these, but I''m fairly sure it can be done. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Tue, 2008-02-12 at 19:05 +0000, Andrew Suffield wrote:> > If you place ''static'' routes in zebra.conf, thusly: > > ip route 10.1.0.1/32 10.1.100.2That works as long as you know ahead of time what your default router is going to be. In the dynamic IP consumer world, that''s not always the case. In fact I can and have had two devices on the same cable modem that get addresses out of entirely different netblocks.> That combined with suitably crafted routing tables should be > sufficient, if I''ve understood your problem correctly.The problem is that the content of those provider tables is dynamic also. As I understand it, each provider table is essentially a copy of the main table (so all known routing entries) minus any entries which are a result of other providers. So given a situation where a non-provider based interface (i.e. VPN) and/or routes show up, the provider tables need to all be updated. Maybe there is a better way to do it than shorewall currently implements. It would indeed be good if the provider tables could somehow only need the relevant entries for what it''s providing. Maybe put the non-provider table before it in the routing rule list. Not really sure. b. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Andrew Suffield wrote:> On Tue, Feb 12, 2008 at 09:46:20AM -0800, Tom Eastep wrote: >> ursa:~ # iptables -L foo >> Chain foo (0 references) >> target prot opt source destination >> bar all -- anywhere anywhere >> ursa:~ # iptables -E bar baz >> ursa:~ # iptables -L foo >> Chain foo (0 references) >> target prot opt source destination >> baz all -- anywhere anywhere >> ursa:~ # > > Which is why it''s not useful here - it has no effect on the structure, > it just changes the name, so it cannot be used as an > atomic-insert-of-many-rules. The necessary feature would be a ''swap > chains'' command, that replaced all references to one existing chain > with references to another existing chain (not that I''m advocating > it). >It is useful -- just not as useful as it might be. If I have chain foo with a jump to bar and I want to replace bar, then I a) create baz as replacement for bar b) populate baz c) Insert jump to baz just before existing jump to bar (with same predicates as the existing jump) d) delete existing jump to bar e) flush bar f) delete bar g) rename baz to bar If I could atomically swap bar and baz, I could omit steps c and d and would not have a small window where packets might go through both bar and baz. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Brian J. Murrell wrote:> > The problem is that the content of those provider tables is dynamic > also. As I understand it, each provider table is essentially a copy of > the main table (so all known routing entries) minus any entries which > are a result of other providers. So given a situation where a > non-provider based interface (i.e. VPN) and/or routes show up, the > provider tables need to all be updated.No. Not if you have a routing rule that directs all traffic to VPN addresses through the main table. That''s why I added the route_rules file in the first place.> > Maybe there is a better way to do it than shorewall currently > implements. It would indeed be good if the provider tables could > somehow only need the relevant entries for what it''s providing. Maybe > put the non-provider table before it in the routing rule list. Not > really sure.The problem is that: a) Provider tables must include a default route so they are always terminating (packets never fall off the end and go to another routing table). b) Packets must be marked *before* they are routed to direct them to a particular provider. So if you mark a packet to associate it with a particular provider then it *will* be routed using that provider''s table unless there is a routing rule before the mark-based ones that sends the packet through a different table. That''s why Shorewall copies routes out of other interfaces (given by the COPY column) into provider tables; it avoids having to have rules which essentially duplicate most of the main table. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Tue, Feb 12, 2008 at 02:16:34PM -0500, Brian J. Murrell wrote:> In the dynamic IP consumer world, that''s not always the > case.I''m not sure there are any sane solutions to routing in that world. Get a better ISP? I haven''t had a dynamic address on anything in over five years...> The problem is that the content of those provider tables is dynamic > also. As I understand it, each provider table is essentially a copy of > the main table (so all known routing entries) minus any entries which > are a result of other providers. So given a situation where a > non-provider based interface (i.e. VPN) and/or routes show up, the > provider tables need to all be updated. > > Maybe there is a better way to do it than shorewall currently > implements. It would indeed be good if the provider tables could > somehow only need the relevant entries for what it''s providing. Maybe > put the non-provider table before it in the routing rule list. Not > really sure.Hmm. I remember thinking about this a couple of years ago and seeing a solution, but it does not immediately come to mind now. I must cogitate further... ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Tue, Feb 12, 2008 at 11:24:16AM -0800, Tom Eastep wrote:> If I have chain foo with a jump to bar and I want to replace bar, then I > > a) create baz as replacement for bar > b) populate baz > c) Insert jump to baz just before existing jump to bar (with same predicates > as the existing jump) > d) delete existing jump to bar > e) flush bar > f) delete bar > g) rename baz to bar > > If I could atomically swap bar and baz, I could omit steps c and d and would > not have a small window where packets might go through both bar and baz.Ah, should have seen that. On reflection, I''m being dim. Here''s the slight modification to give an atomic solution with the current features: (starting from default empty tables) iptables -N foo iptables -N foo_ref iptables -A INPUT -j foo_ref iptables -A foo_ref -j foo (populate foo with your rules) Atomic switch operation: iptables -N new_foo (populate new_foo) iptables -I foo_ref 1 -j new_foo iptables -D foo_ref 2 iptables -X foo iptables -E new_foo foo The trick is that the insert of a new unconditional jump in foo_ref is itself an atomic operation, which renders the old jump inactive simply by position. This does, of course, require that the chain foo never ''-j RETURN''s. (There''s still no way to atomically modify all the root chains at once) ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Tue, 2008-02-12 at 19:37 +0000, Andrew Suffield wrote:> > I''m not sure there are any sane solutions to routing in that > world.Indeed.> Get a better ISP?The problem is not a bad singular ISP. It''s combining more than one ISP that starts to make the problem. And the bottom line is that for most of the world, ISP shopping is not like going to the supermarket and having forty-eleven brands to choose from. I''m lucky I have a few choices, but there are people in rural parts that have 1 choice (but they don''t apply here) or perhaps two choices and if they want multiple links, that means no choice. Sure, in an ideal world, ISPs would operate sanely. We are far from that and must deal in any case.> Hmm. I remember thinking about this a couple of years ago and seeing a > solution, but it does not immediately come to mind now. I must > cogitate further...Part of the problem is the inflexibility of various players. Everything that deals with routing assumes a single "main" routing table when in a more complex world that''s not the case and there is no ability to step in and change that. That''s why shorewall''s "hack"s to deal with it is used by the people who use it. b. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Brian J. Murrell wrote:> > Part of the problem is the inflexibility of various players. Everything > that deals with routing assumes a single "main" routing table when in a > more complex world that''s not the case and there is no ability to step > in and change that. That''s why shorewall''s "hack"s to deal with it is > used by the people who use it. >My what you miss when your sleeping... If your talking init scripts here, right? The issues I found with dealing with iproute2, is that the packages that deal with obtaining/creating an ip address only assume the main routing table should be used. While what we really want here, I think, is one table per ip, maybe one per net route with secondary ips using src. Much like what a table looks like with what shorewall generates without an entry in the copy column for a provider, just the ip specific routing. Then the other issue becomes /etc/iproute2 data, what should the table names be? Tom''s manipulation of rt_tables could be used to set the table''s name based on the interface name, and the related route rule code use to create the routing rule. Thus, to look up a route it would be: ip route ls table ppp0, ip route ls table tun0, ip route ls table eth0 etc... I''m I on track here or way off base? To change the networking scripts would be a major under taking here, and maintaining patches until upstream came on board would be a headache, I gave up with the little bit I came up with for multi-dhcp provider support for fedora. Maybe, I''m up for it now, if there is others who are interested, otherwise I''m not going to bother trying to change the world. Jerry ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Hello gurus... If I have 2 internet connections and I like to have one as redundante to the other, how I manage them with shorewall? Thanks, Daniel ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Daniel Monza wrote:> Hello gurus... > > If I have 2 internet connections and I like to have one as redundante to the > other, how I manage them with shorewall?http://www.shorewall.net/MultiISP.html But beware -- given that Shorewall isn''t something that runs continuously in your system, there is no automatic failover. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
On Tue, 2008-02-12 at 19:29 -0600, Jerry Vonau wrote:> > My what you miss when your sleeping... > > If your talking init scripts here, right?Well, initscripts in terms of any of /etc/init.d, /sbin/dhclient-script /etc/ppp/ip-up[.d] and so forth, yes.> The issues I found with > dealing with iproute2, is that the packages that deal with > obtaining/creating an ip address only assume the main routing table > should be used._Exactly!_ Even routing protocols like quagga only deal with the single main routing table -- AFAIK.> While what we really want here, I think, is one table > per ip,IP or interface?> maybe one per net route with secondary ips using src. Much like > what a table looks like with what shorewall generates without an entry > in the copy column for a provider, just the ip specific routing. Then > the other issue becomes /etc/iproute2 data, what should the table names > be? Tom''s manipulation of rt_tables could be used to set the table''s > name based on the interface name, and the related route rule code use to > create the routing rule. > Thus, to look up a route it would be: ip route ls table ppp0, > ip route ls table tun0, ip route ls table eth0 etc...Hrmmmm.> I''m I on track here or way off base?I''m not sure. I''d have to flesh out an example or see an example to know for sure.> To change the networking scripts would be a major under taking here, > and maintaining patches until upstream came on board would be a > headache,Agreed.> I gave up with the little bit I came up with for multi-dhcp > provider support for fedora. Maybe, I''m up for it now, if there is > others who are interested, otherwise I''m not going to bother trying to > change the world.It seems to me that what we really want is configurable route manipulation. Something like a set of rules and filters between "route add" (and quagga and whatever else will want to manipulate routing) and the final routing manipulation. The idea would be a process that receives input in the form of a userspace routing request (source, dest, interface, etc.) that goes through user configurable filters and rules before it is finally inserted (or deleted or replaced as the case may be) into one or more routing tables. Or am I way off base? b. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Brian J. Murrell wrote: Hey you updated the /sbin/dhclient-script in 1999.> On Tue, 2008-02-12 at 19:29 -0600, Jerry Vonau wrote: >> My what you miss when your sleeping... >> >> If your talking init scripts here, right? > > Well, initscripts in terms of any > of /etc/init.d, /sbin/dhclient-script /etc/ppp/ip-up[.d] and so forth, > yes. > >> The issues I found with >> dealing with iproute2, is that the packages that deal with >> obtaining/creating an ip address only assume the main routing table >> should be used. > > _Exactly!_ Even routing protocols like quagga only deal with the single > main routing table -- AFAIK. >pppd is easy enough to fix with the ip-up.local file. /sbin/dhclient-script is fixable, then comes the rest of the network functions. What really is needed is some convention regarding table names, you can use whatever you want in rt_tables, hence the suggestion below. Once that is in place, at least porting the rest of the scripts won''t be as bad, you''ll know in advance what the advanced table name should be.>> While what we really want here, I think, is one table >> per ip, > > IP or interface? > >> maybe one per net route with secondary ips using src. Much like >> what a table looks like with what shorewall generates without an entry >> in the copy column for a provider, just the ip specific routing.Not interface, you could have more that one network per interface. Need to play around a bit, not sure if that should be one per ip or per network gateway.>> Then >> the other issue becomes /etc/iproute2 data, what should the table names >> be? Tom''s manipulation of rt_tables could be used to set the table''s >> name based on the interface name, and the related route rule code use to >> create the routing rule. >> Thus, to look up a route it would be: ip route ls table ppp0, >> ip route ls table tun0, ip route ls table eth0 etc... > > Hrmmmm. > >> I''m I on track here or way off base? > > I''m not sure. I''d have to flesh out an example or see an example to > know for sure. >Quick test, use the interface name as the ISP name in the providers file. I''d be looking to handle the rt_tables/rules in the same way that the providers'' functions do, just a different sourcing for the names.>> To change the networking scripts would be a major under taking here, >> and maintaining patches until upstream came on board would be a >> headache, > > Agreed. > >> I gave up with the little bit I came up with for multi-dhcp >> provider support for fedora. Maybe, I''m up for it now, if there is >> others who are interested, otherwise I''m not going to bother trying to >> change the world. > > It seems to me that what we really want is configurable route > manipulation. Something like a set of rules and filters between "route > add" (and quagga and whatever else will want to manipulate routing) and > the final routing manipulation. The idea would be a process that > receives input in the form of a userspace routing request (source, dest, > interface, etc.) that goes through user configurable filters and rules > before it is finally inserted (or deleted or replaced as the case may > be) into one or more routing tables. > > Or am I way off base? >I think that is a stop gap hack, another file to edit... I believe that the correct way would be to create the routing table/rules when the interface is created, then port whatever else your using to change the routing to use the advanced tables. See what I mean about changing the world. That might be a very big task at first or it might be just a few lines here and there, but the rewards might be worth it . Jerry ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/