Hi All, As you know, I''ve discussed here a number of times in the past about the problems with the MultiISP support in shorewall, specifically in the way routing is set up. I wonder if we can restructure the route rules to better accommodate the after-shorewall-has-started-route-manipulation problem. To provide a real-world use case, imagine a MultiISP shorewall and openvpn configuration where the multiple ISP links are tracked and balanced. This all works fine as long as nothing comes along and adds routes to the "main" routing table after shorewall has made it''s per-provider copy of it. So this means of course that openvpn, if it''s going to install client subnet routes, needs to be run first. It also means that if openvpn gets some updated configuration, shorewall needs to be restarted to get the additions in it''s per-provider routing tables. So, can we rearrange the route rules, possibly adding more rules and tables to somehow eliminate this need to copy the main table for the per-provider tables and taking advantage of the fact that all of the routing tools out there, unless they are capable and have been told otherwise, add to the "main" table. I''m still trying to wrap my head around it, but I''m thinking of something along the lines of not having any default routing at all in the main table (all interface plumbing processes like PPP, DHCP, etc. need to be told not to plumb default routes), having the main table before the per-provider tables and having the per provider tables be not much more, if anything at all than default routes through their interfaces and then a catchall default table at the end. So the route rules would look like: 0: from all lookup local 32766: from all lookup main 40000: from all fwmark 0x100 lookup ISP1 40001: from all fwmark 0x200 lookup ISP2 50000: from 167.3.32.3 lookup ISP1 50256: from 166.1.37.24 lookup ISP2 65535: from all lookup default table ISP1 would look like: 167.3.32.3 dev eth0.1 scope link default via 167.3.208.1 dev eth0.1 table ISP2 would be the same, but for it''s provider table main would be the standard default routing table, with NO default routes: 213.197.29.32 via 167.3.208.1 dev eth0.1 213.204.193.2 via 167.3.208.1 dev eth0.1 10.8.0.2 dev tun0 proto kernel scope link src 10.8.0.1 192.168.200.1 dev ppp0 proto kernel scope link src 166.1.37.24 10.8.0.0/24 via 10.8.0.2 dev tun0 192.168.1.0/24 via 10.75.22.1 dev br-lan proto zebra metric 20 equalize 192.168.0.0/24 via 10.75.22.5 dev br-lan proto zebra equalize 10.75.22.0/24 dev br-lan proto kernel scope link src 10.75.22.199 10.75.23.0/24 via 10.8.0.2 dev tun0 167.3.208.0/20 dev eth0.1 proto kernel scope link src 167.3.32.3 169.254.0.0/16 via 10.75.22.223 dev br-lan proto zebra metric 20 equalize and then routing table "default" would have the load balancing routes: default nexthop via 167.3.208.1 dev eth0.1 weight 1 nexthop via 192.168.200.1 dev ppp0 weight 1 Does something like this have any chance of working or am I totally on crack? The one big caveat is of course, that processes which plumb interfaces such as DHCP, PPP, etc. CANNOT plumb default routes and Shorewall has to take care of that. That''s how I run my MultiISP router now anyway though, so it''s doable. It also seems that the 50* route rules are not needed/useful with this setup. TBH, I''m not sure why they are needed even before my proposed changes. Maybe their purpose will be the downfall of my proposal. Thots? b. ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php
Brian J. Murrell wrote:> > To provide a real-world use case, imagine a MultiISP shorewall and > openvpn configuration where the multiple ISP links are tracked and > balanced. This all works fine as long as nothing comes along and adds > routes to the "main" routing table after shorewall has made it''s > per-provider copy of it. So this means of course that openvpn, if it''s > going to install client subnet routes, needs to be run first.As I''ve tried to explain on multiple occasions, *it does not mean that*. If your OpenVPN server is going to add routes to hosts in the 192.168.2.0/24 network then simply add this line to your route_rules file: - 192.168.2.0/24 254 1001 Solving the OpenVPN routing problem was one of the main reasons for creating the route_rules file in the first place. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php
On Sun, 2008-06-29 at 10:47 -0700, Tom Eastep wrote:> > If your OpenVPN server is going to add routes to hosts in the 192.168.2.0/24 > network then simply add this line to your route_rules file: > > - 192.168.2.0/24 254 1001 > > Solving the OpenVPN routing problem was one of the main reasons for creating > the route_rules file in the first place.Yes, a table before the provider tables was the other solution I was thinking of. I was looking to solve the more general problem of routing-happens-in-the-main table. OpenVPN was just the example I had on hand. I guess in general, I''d just like to see shorewall work better with dynamic routing than it does -- with less before-hand preparation. Do you think my proposed routing rules/tables reorganization would not achieve that, or not work at all? b. ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php
Brian J. Murrell wrote:> Do you think my proposed routing rules/tables reorganization would not > achieve that, or not work at all?I believe that it doesn''t work at all. You have to know where a packet is going before you mark it. Try creating marking rules that work in your scheme where the firewall has a DMZ, a LOC zone and a two NET interfaces and you will see what I mean. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php
On Sun, 2008-06-29 at 14:06 -0700, Tom Eastep wrote:> > I believe that it doesn''t work at all. You have to know where a packet is > going before you mark it.Does changing up the order of the routing table traversal change that? Actual marking is done in the mangle table right? In my case for example I have: Chain routemark (2 references) pkts bytes target prot opt in out source destination 3484 458K MARK all -- ppp0 * 0.0.0.0/0 0.0.0.0/0 MARK set 0x200 21601 3378K MARK all -- eth0.1 * 0.0.0.0/0 0.0.0.0/0 MARK set 0x100 25085 3836K CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 MARK match !0x0/0xff00 CONNMARK save mask 0xff00 But that''s based on source interfaces, which I don''t think routing table/rule reorganization would alter.> Try creating marking rules that work in your > scheme where the firewall has a DMZ, a LOC zone and a two NET interfaces and > you will see what I mean.Unfortunately I don''t easily have a testbed where I can do that easily and I can''t say that I''ve seen what shorewall will do to the routing and marking rules in that case. It would be interesting to look at though, indeed. Anyone have such a configuration you can send me the output of your routing rules, tables and mangle table? b. ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php
Brian J. Murrell wrote:> On Sun, 2008-06-29 at 14:06 -0700, Tom Eastep wrote: >> I believe that it doesn''t work at all. You have to know where a packet is >> going before you mark it. > > Does changing up the order of the routing table traversal change that? > Actual marking is done in the mangle table right? In my case for > example I have: > > Chain routemark (2 references) > pkts bytes target prot opt in out source destination > 3484 458K MARK all -- ppp0 * 0.0.0.0/0 0.0.0.0/0 MARK set 0x200 > 21601 3378K MARK all -- eth0.1 * 0.0.0.0/0 0.0.0.0/0 MARK set 0x100 > 25085 3836K CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 MARK match !0x0/0xff00 CONNMARK save mask 0xff00 > > But that''s based on source interfaces, which I don''t think routing > table/rule reorganization would alter.Those are Shorewall-generated rules from specifying ''track'' on your providers. Those won''t work under your scheme because your provider routing tables wouldn''t have a route to your local network or to your DMZ!> >> Try creating marking rules that work in your >> scheme where the firewall has a DMZ, a LOC zone and a two NET interfaces and >> you will see what I mean. > > Unfortunately I don''t easily have a testbed where I can do that easily > and I can''t say that I''ve seen what shorewall will do to the routing and > marking rules in that case. It would be interesting to look at though, > indeed. Anyone have such a configuration you can send me the output of > your routing rules, tables and mangle table?Just something simple like "I want all SMTP traffic to go out of provider 1 (with mark 1)" is not straightforward under your scheme. You would like to add this marking rule: 1:P 0.0.0.0/0 0.0.0.0/0 tcp 25 But suppose that you have an mail server in your DMZ. Packets marked by that rule will be sent through provider 1''s routing table *which doesn''t have a route to your DMZ". So you find yourself having to replicate your routing in your marking rules: 1:P 0.0.0.0/0 !xxx.xxx.xxx.xxx tcp 25 where xxx.xxx.xxx.xxx is the IP address of your DMZ mail server. I think that adding routing rules for things that need to use the main table is more straightforward. Plus your scheme would be incompatible with current configurations. Which is also a problem. Because that means that we get to support both the current scheme and your scheme. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php
On Mon, 2008-06-30 at 16:45 -0700, Tom Eastep wrote:> > I think that adding routing rules for things that need to use the main table > is more straightforward.Indeed, if you know ahead of time and are able/willing to reload your firewall to deal with changes.> Plus your scheme would be incompatible with current > configurations. Which is also a problem. Because that means that we get to > support both the current scheme and your scheme.Yeah. I''m just looking for something that works more naturally with the world view that routing happens in the "main" table. b. ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php
Brian J. Murrell wrote:> On Mon, 2008-06-30 at 16:45 -0700, Tom Eastep wrote: >> I think that adding routing rules for things that need to use the main table >> is more straightforward. > > Indeed, if you know ahead of time and are able/willing to reload your > firewall to deal with changes.Brian, You''re being deliberately dense. There is no reason to ''reload your firewall to deal with the changes''. You only have to understand the nature of the changes that your applications might apply to your main table and supply routing rules *in advance* that anticipate those changes. I have been down this road many times; I''ve been thinking about the problem a lot longer than you have and I don''t believe that there is a simple answer. I''m still willing to be convinced; but the ''provider tables contain only default routes'' approach is a dead end as far as I''m able to see. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php
On Mon, 2008-06-30 at 20:45 -0700, Tom Eastep wrote:> Brian J. Murrell wrote: > > On Mon, 2008-06-30 at 16:45 -0700, Tom Eastep wrote: > >> I think that adding routing rules for things that need to use the main table > >> is more straightforward. > > > > Indeed, if you know ahead of time and are able/willing to reload your > > firewall to deal with changes. > > Brian, > > You''re being deliberately dense.I think in fact just failing to communicate effectively.> There is no reason to ''reload your firewall > to deal with the changes''.As long as I have already planned for them in the route_rules.> You only have to understand the nature of the > changes that your applications might apply to your main table and supply > routing rules *in advance* that anticipate those changes.Right. Understood completely. My "reload your firewall" comment was in response to the scenario where something I had not planned for ahead of time comes along and thus I have to update the route_rules configuration.> I have been down this road many times; I''ve been thinking about the problem > a lot longer than you have and I don''t believe that there is a simple > answer.Indeed. And I recognize that, which is why I put my idea up for criticism. I knew if there were flaws with it, I could count on the collective experience here to find them.> I''m still willing to be convinced; but the ''provider tables contain > only default routes'' approach is a dead end as far as I''m able to see.Yeah, it very well could be. I do recognize you are the word of experience here. But sometimes even hairbrained proposals sometimes make the experienced people think in ways or about solutions they had not considered before. I''ll keep thinking about it. :-) b. ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php
On Tue, 2008-07-01 at 07:12 -0700, Tom Eastep wrote:> Brian J. Murrell wrote: > > On Mon, 2008-06-30 at 20:45 -0700, Tom Eastep wrote: > >> I'm still willing to be convinced; but the 'provider tables contain > >> only default routes' approach is a dead end as far as I'm able to see. > > > > Yeah, it very well could be. I do recognize you are the word of > > experience here. But sometimes even hairbrained proposals sometimes > > make the experienced people think in ways or about solutions they had > > not considered before. I'll keep thinking about it. :-) > > Brian, > > I owe you an apology. I missed (or kept ignoring) the essential feature of > your proposal that *does* allow it to work; namely the way in which you > re-ordered the routing rules. I awoke this morning with the realization that > your proposal would work with the right rule ordering and when I looked at > your original post, there it was. I'm truly sorry for being so > dense/stubborn/whatever. > > So given that it can work, we need to decide what to do about it. I really > dislike the notion of two models for routing but I suspect that is the only > way in which I could implement this scheme without causing serious > compatibility issues. More thought needed. > > -TomOK, for those of us that are playing along at home ;-), to condense the thought, what we(?) would be looking at is a single "bal" table that has the default routes. The routing rules needed would point to the "main" routing table for the routes that would be "local" to the box (invert the logic, ie: ip rule to 10.3.0.10/24 lookup table main), while the routes via an isp that are "external" to the box would be directed to the "bal" (default?) table, (ie: ip rule to 0.0.0.0/0 lookup table bal), with the "ip rules" ordering winning the table race. I wonder if that is what the stock blank "default" table is meant for? (vpn routes would be considered local here). I like this, it *should* work kind of like the squid routing, point to a gateway(s) and the rest should just fall into line(with the routing rules in place), with much less code perhaps. Have you thought about what the routing rules might look like in this setup? Just my thoughts, Jerry ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
Brian J. Murrell wrote:> On Mon, 2008-06-30 at 20:45 -0700, Tom Eastep wrote: >> I''m still willing to be convinced; but the ''provider tables contain >> only default routes'' approach is a dead end as far as I''m able to see. > > Yeah, it very well could be. I do recognize you are the word of > experience here. But sometimes even hairbrained proposals sometimes > make the experienced people think in ways or about solutions they had > not considered before. I''ll keep thinking about it. :-)Brian, I owe you an apology. I missed (or kept ignoring) the essential feature of your proposal that *does* allow it to work; namely the way in which you re-ordered the routing rules. I awoke this morning with the realization that your proposal would work with the right rule ordering and when I looked at your original post, there it was. I''m truly sorry for being so dense/stubborn/whatever. So given that it can work, we need to decide what to do about it. I really dislike the notion of two models for routing but I suspect that is the only way in which I could implement this scheme without causing serious compatibility issues. More thought needed. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php
On Tue, 2008-07-01 at 07:12 -0700, Tom Eastep wrote:> > Brian, > > I owe you an apology.No worries. I was not totally convinced myself that it would work (or I would have been more persistent, but to have been more that it would work I would have had to find the time to do an implemenation) and was hoping for verification/critique by those more experienced than I.> I missed (or kept ignoring) the essential feature of > your proposal that *does* allow it to work; namely the way in which you > re-ordered the routing rules.Excellent.> I awoke this morning with the realization that > your proposal would work with the right rule ordering and when I looked at > your original post, there it was.:-)> I''m truly sorry for being so > dense/stubborn/whatever.Again, no worries. I had not totally given up the idea but I knew I had put my money where my mouth was and actually try it out to see for myself why it would or wouldn''t work.> So given that it can work, we need to decide what to do about it. I really > dislike the notion of two models for routing but I suspect that is the only > way in which I could implement this scheme without causing serious > compatibility issues. More thought needed.Is it "two models" or just a re-implementation of the existing model? What if the only change was to do the route rules re-ordering so that applications populating the main table would get what they want? Does anything "user visible" (i.e. anything in /etc/shorewall/) really need to change? route_rules could even still be functional, just not needed as much (there might still be corner cases) or at all. I guess there is the slight user visible change that they have to ensure that interface plumbing processes don''t plumb a default route. Then again, shorewall could always just [re]move them [to the default table]. I think it''s generally a requirement that shorewall be reloaded when interfaces go up and down anyway. b. ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
Jerry Vonau wrote:> > OK, for those of us that are playing along at home ;-), to condense the > thought, what we(?) would be looking at is a single "bal" table that has > the default routes. The routing rules needed would point to the "main" > routing table for the routes that would be "local" to the box (invert > the logic, ie: ip rule to 10.3.0.10/24 lookup table main), while the > routes via an isp that are "external" to the box would be directed to > the "bal" (default?) table, (ie: ip rule to 0.0.0.0/0 lookup table bal), > with the "ip rules" ordering winning the table race.Exactly.>I wonder if that > is what the stock blank "default" table is meant for? (vpn routes would > be considered local here).I suspect so.> I like this, it *should* work kind of like > the squid routing, point to a gateway(s) and the rest should just fall > into line(with the routing rules in place), with much less code perhaps. > Have you thought about what the routing rules might look like in this > setup?Attached is a copy of what I have running currently. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
Brian J. Murrell wrote:> > Is it "two models" or just a re-implementation of the existing model? > What if the only change was to do the route rules re-ordering so that > applications populating the main table would get what they want? Does > anything "user visible" (i.e. anything in /etc/shorewall/) really need > to change? route_rules could even still be functional, just not needed > as much (there might still be corner cases) or at all.Consider the case of a transparent Squid proxy in the local net. The recommended rule there is #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS Squid 1 202 - eth1 192.168.1.3 loose Packets with mark 202 are sent to 192.168.1.3 regardless of the destination IP address. Under the new scheme (I''m currently calling the option ROUTING_NG), packets with mark 202 are sent to 192.168.1.3 *only if there is no route to the destination IP address in the main routing table*. So the new behavior is definitely different and incompatible with the old behavior.> > I guess there is the slight user visible change that they have to ensure > that interface plumbing processes don''t plumb a default route. Then > again, shorewall could always just [re]move them [to the default table]. > I think it''s generally a requirement that shorewall be reloaded when > interfaces go up and down anyway.Yes. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
On Tue, 2008-07-01 at 11:59 -0700, Tom Eastep wrote:> > Consider the case of a transparent Squid proxy in the local net.Indeed. This is one use-case I''ve never played with. I''m surprised to discover that it''s achieved using a "provider", but can see how/why.> The > recommended rule there is > > #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS > Squid 1 202 - eth1 192.168.1.3 loose > > Packets with mark 202 are sent to 192.168.1.3 regardless of the destination > IP address. Under the new scheme (I''m currently calling the option > ROUTING_NG), packets with mark 202 are sent to 192.168.1.3 *only if there is > no route to the destination IP address in the main routing table*.Indeed, because the packet needs to pass through the main table before it will get to a provider table.> So the new behavior is definitely different and incompatible with the old > behavior.I wonder if a new field (yeah, not terribly desirable, but we are proposing removing a field at the same time) to the providers table to flag whether the provider is subject to the main table or overrides it. b. ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
Brian J. Murrell wrote:> >> So the new behavior is definitely different and incompatible with the old >> behavior. > > I wonder if a new field (yeah, not terribly desirable, but we are > proposing removing a field at the same time)No, we aren''t -- we are proposing to require that two fields (DUPLICATE and COPY) be empty when the new behavior applies. Brian, we can''t just remove columns from configuration files between releases. Users often upgrade Shorewall as part of a distribution upgrade where hundreds of packages are changing. We can''t make incompatible changes that result in their firewall not starting after the upgrade.> to the providers table to > flag whether the provider is subject to the main table or overrides it.I''ve thought of that approach (adding a provider option) but what happens if part of the entries have the new option and part don''t? If you can figure out what should happen, then do you want to be responsible for documenting it as well? -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
On Tue, 2008-07-01 at 18:57 -0700, Tom Eastep wrote:> > No, we aren''t -- we are proposing to require that two fields (DUPLICATE and > COPY) be empty when the new behavior applies.Ahhh. OK. I see.> Brian, we can''t just remove > columns from configuration files between releases. Users often upgrade > Shorewall as part of a distribution upgrade where hundreds of packages are > changing. We can''t make incompatible changes that result in their firewall > not starting after the upgrade.Yes, that''s a very valid point.> I''ve thought of that approach (adding a provider option)You mean a new field/option to the provider table, yes?> but what happens if > part of the entries have the new option and part don''t?s/part/some/ i.e. so some providers entries have the "override main routing" and some don''t? I think that case is actually the more clear case. In that case, we can deduce that the user has actually chosen to use the new functionality. The more unclear case to me is that no provider entries have the new field selected. Is this because the user''s preferences are that none of the providers override the main table or that they simply have not recognized the new feature and made adjustments to reflect it? I wonder if we could/should have a shorewall.conf feature (default to off of course) for this new behaviour. At least through a transition to a new major release that requires this new behaviour. Or do your release standards forbid any new release to require user intervention to adopt some new/changed functionality? That''s fair enough if they do. I can certainly see a good point for that kind of policy. I''m just at the moment unclear if that''s the case. b. ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
Brian J. Murrell wrote:> >> I''ve thought of that approach (adding a provider option) > > You mean a new field/option to the provider table, yes?Yes -- a new option.> >> but what happens if >> part of the entries have the new option and part don''t? > > s/part/some/ > > i.e. so some providers entries have the "override main routing" and some > don''t? I think that case is actually the more clear case. In that > case, we can deduce that the user has actually chosen to use the new > functionality.The issue is not trying to figure out what the user wants but rather what should happen. We can''t leave the user''s default route(s) in the main table; about all we can do is to try to move it (them) to the default table, I guess.> > I wonder if we could/should have a shorewall.conf feature (default to > off of course) for this new behaviour.That''s what my current prototype does (option is named ROUTING_NG but I''m not particular happy with that name). But I think that is the safest thing to do.> At least through a transition to > a new major release that requires this new behaviour. Or do your > release standards forbid any new release to require user intervention to > adopt some new/changed functionality?We generally *require* the user to explicitly enable new functionality (no gain, no pain). One thing that bothers me about this whole thing is that it trades one sharp edge for another. In the current scheme, applications that add non-default routes to the main table are a problem; although it is the application itself that doesn''t work, not the router as a whole. In the ROUTING_NG configuration, having a default route unexpectedly added to the main table is a disaster; it can isolate the firewall/router entirely. I''m not sure that I want to give users that much rope to hang themselves. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
On Wed, 2008-07-02 at 07:05 -0700, Tom Eastep wrote:> > The issue is not trying to figure out what the user wants but rather > what should happen. We can''t leave the user''s default route(s) in the > main table; about all we can do is to try to move it (them) to the > default table, I guess.If they choose to use the ROUTING_NG option, yes. I''d posit that selecting ROUTING_NG and finding default routes in the main table is in fact a configuration error! ROUTING_NG requires that default route plumbing by interface configuration tools be disabled, yes? None of that covers the case where the default routes appear in the main table after shorewall has done it''s business of course.> We generally *require* the user to explicitly enable new functionality > (no gain, no pain).Indeed.> One thing that bothers me about this whole thing is that it trades one > sharp edge for another. In the current scheme, applications that add > non-default routes to the main table are a problem; although it is the > application itself that doesn''t work, not the router as a whole.True.> In the > ROUTING_NG configuration, having a default route unexpectedly added to > the main table is a disaster; it can isolate the firewall/router > entirely.Well, it wouldn''t isolate it off of any local networks, but yes, it could certainly foul up the provider routing that''s supposed to happen.> I''m not sure that I want to give users that much rope to hang > themselves.Even with a shorewall.conf option and a blurb about disabling default route plumbing in their interface configuration mechanism, and a check for "default" in the main table (yielding a complete failure to install the configuration) at rule installation time? Heh. It''s almost like one needs to be able to apply filters to routing tables, preventing matching routes from being entered into them. Or a routing management interface such as we have discussed here before. b. ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
On Sat, 2008-07-05 at 08:46 -0700, Tom Eastep wrote:> Brian J. Murrell wrote: > > On Wed, 2008-07-02 at 07:05 -0700, Tom Eastep wrote: > > > > >> I''m not sure that I want to give users that much rope to hang > >> themselves. > > For those who are brave, there is a preview of Beta3 available at > http://www.shorewall.net/pub/shorewall/development/staging/4.2/shorewall-4.2.0-Beta3/. >Cool, thanks, it will be a couple of days before I can play around, and offer any feedback.> I decided to call the option ROUTE_BALANCE although I can still be > talked into another name.How about "USE_DEFAULT_TABLE", or maybe "GW_IN_DEFAULT_TABLE"?> See the release notes for a description of how > it works. The Multi-ISP doc included with the release also described the > feature. I''m going to mark the feature ''Experimental'' until we really > understand its implications. > > -TomJerry ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
Brian J. Murrell wrote:> On Wed, 2008-07-02 at 07:05 -0700, Tom Eastep wrote:> >> I''m not sure that I want to give users that much rope to hang >> themselves.For those who are brave, there is a preview of Beta3 available at http://www.shorewall.net/pub/shorewall/development/staging/4.2/shorewall-4.2.0-Beta3/. I decided to call the option ROUTE_BALANCE although I can still be talked into another name. See the release notes for a description of how it works. The Multi-ISP doc included with the release also described the feature. I''m going to mark the feature ''Experimental'' until we really understand its implications. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
Jerry Vonau wrote:> On Sat, 2008-07-05 at 08:46 -0700, Tom Eastep wrote: > >> I decided to call the option ROUTE_BALANCE although I can still be >> talked into another name. > > How about "USE_DEFAULT_TABLE", or maybe "GW_IN_DEFAULT_TABLE"? >USE_DEFAULT_TABLE would work. Thanks, -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
On Tue, 2008-07-01 at 11:17 -0700, Tom Eastep wrote:> Jerry Vonau wrote: > > > > > OK, for those of us that are playing along at home ;-), to condense the > > thought, what we(?) would be looking at is a single "bal" table that has > > the default routes. The routing rules needed would point to the "main" > > routing table for the routes that would be "local" to the box (invert > > the logic, ie: ip rule to 10.3.0.10/24 lookup table main), while the > > routes via an isp that are "external" to the box would be directed to > > the "bal" (default?) table, (ie: ip rule to 0.0.0.0/0 lookup table bal), > > with the "ip rules" ordering winning the table race. > > Exactly. > > >I wonder if that > > is what the stock blank "default" table is meant for? (vpn routes would > > be considered local here). > > I suspect so. > > > I like this, it *should* work kind of like > > the squid routing, point to a gateway(s) and the rest should just fall > > into line(with the routing rules in place), with much less code perhaps. > > Have you thought about what the routing rules might look like in this > > setup? > > Attached is a copy of what I have running currently. > > -TomI moved my default gateway from the main table to the default table on an otherwise out of the box fedora9 box, I'm still on the net. :-) shorewall show routing Shorewall 4.0.12 Routing at S010600e029961c55 - Mon Jul 7 14:03:41 CDT 2008 Routing Rules 0: from all lookup local 32766: from all lookup main 32767: from all lookup default Table default: default via 24.76.252.1 dev eth0 Table local: broadcast 127.255.255.255 dev lo proto kernel scope link src 127.0.0.1 local 24.76.255.80 dev eth0 proto kernel scope host src 24.76.255.80 broadcast 24.76.255.255 dev eth0 proto kernel scope link src 24.76.255.80 broadcast 24.76.252.0 dev eth0 proto kernel scope link src 24.76.255.80 broadcast 127.0.0.0 dev lo proto kernel scope link src 127.0.0.1 local 127.0.0.1 dev lo proto kernel scope host src 127.0.0.1 local 127.0.0.0/8 dev lo proto kernel scope host src 127.0.0.1 Table main: 24.76.252.0/22 dev eth0 proto kernel scope link src 24.76.255.80 169.254.0.0/16 dev eth0 scope link The routing rules are in the same order, just with different values, I'm wondering if the "from <ip> lookup <table>" rules are even need/wanted. When a connection is from the fw to a host that is on the same lan as a gateway, I not sure with out testing, if that would mess up the the ip rule lookup for that target's ip, given that there is no route in the providers table, other that the host route to the gateway, or would an earlier ip rule cover it? (OK, I'm a bit rusty...) This looks promising, I'll try my dual-isp box tomorrow, with the beta. Jerry ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
Jerry Vonau wrote:> The routing rules are in the same order, just with different values, I''m > wondering if the "from <ip> lookup <table>" rules are even need/wanted. > When a connection is from the fw to a host that is on the same lan as a > gateway, I not sure with out testing, if that would mess up the the ip > rule lookup for that target''s ip, given that there is no route in the > providers table, other that the host route to the gateway, or would an > earlier ip rule cover it? (OK, I''m a bit rusty...)If there is a route out of the interface for other networks, then the packet will be routed by the main table. By the time we have passed through the main table, the packet is going to be routed by a default route. Those "<from <ip> lookup <table> rules" are there to make clients that bind to a particular local IP work reasonably. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
Tom Eastep wrote:> Jerry Vonau wrote: > >> >> OK, for those of us that are playing along at home ;-), to condense the >> thought, what we(?) would be looking at is a single "bal" table that has >> the default routes. The routing rules needed would point to the "main" >> routing table for the routes that would be "local" to the box (invert >> the logic, ie: ip rule to 10.3.0.10/24 lookup table main), while the >> routes via an isp that are "external" to the box would be directed to >> the "bal" (default?) table, (ie: ip rule to 0.0.0.0/0 lookup table bal), >> with the "ip rules" ordering winning the table race. > > Exactly. >Nice to know I might still have it. ;-)>> I wonder if that >> is what the stock blank "default" table is meant for? (vpn routes would >> be considered local here). > > I suspect so. >That would be an easier sell upstream, since it's present but not used, for changes to dhcp, pppd, etc.., that would need to have the table to add the default route to "fixed".>> I like this, it *should* work kind of like >> the squid routing, point to a gateway(s) and the rest should just fall >> into line(with the routing rules in place), with much less code perhaps. >> Have you thought about what the routing rules might look like in this >> setup? > > Attached is a copy of what I have running currently. > > -TomRouting Rules 0: from all lookup local 32766: from all lookup main 35000: from all to 206.124.146.179 lookup linksys 40000: from all fwmark 0x1 lookup linksys 40001: from all fwmark 0x2 lookup shorewall 50000: from 172.20.1.102 lookup linksys 50256: from 192.168.1.5 lookup shorewall 65535: from all lookup default Wow, is that utterly simple, lookup the local rule based on ip/fwmark, if a route is found in that table use it, if not, check the default table for a gateway. I really like this layout. Ok, is been more that a couple of days. ;-) With 4.2, is the reason behind the shorewall test layout, using main 999, is for backwards compatibility? Getting the "squid in loc" to work with "loose" took a bit of effort but that works now. Give me a bit, I'll have some config info that worked for me if you want. I've updated my patches from 2005(? boy, things that you thought were important...) for this layout, for use with Fedora. I have some basic patches, that may need a bit of work, for dhclient-script, network-functions, and eth-up parsing the ifcfg files looking for table/weight info working like the providers functions. While pppd would need to use nodefaultgateway and an bit of ip-up magic to work, but I can't play, no dsl and no time atm. One a side note: Running /sbin/iptables-restore... iptables-restore v1.4.1.1: host/network `!' not found Error occurred at line: 134 Try `iptables-restore -h' or 'iptables-restore --help' for more information. ERROR: iptables-restore Failed. Input is in /var/lib/shorewall/.iptables-restore-input Line 134: -A loc2fw -p 6 --dport 8080 -m conntrack --ctorigdst ! 10.3.0.10 -j ACCEPT editing out this line in rules allows a start: REDIRECT loc 8080 tcp 80 - !10.3.0.10 Did I miss something along the journey? A bit of retooling got things to work. Off to Mom's, just my 2cents, Jerry ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
Jerry Vonau wrote:> > Ok, is been more that a couple of days. ;-) With 4.2, is the reason > behind the shorewall test layout, using main 999, is for backwards > compatibility?Yes.> > Getting the "squid in loc" to work with "loose" took a bit of effort but > that works now. Give me a bit, I''ll have some config info that worked > for me if you want.Please -- I haven''t tested that configuration.> > One a side note: > > Running /sbin/iptables-restore... > iptables-restore v1.4.1.1: host/network `!'' not found > Error occurred at line: 134 > Try `iptables-restore -h'' or ''iptables-restore --help'' for more information. > ERROR: iptables-restore Failed. Input is in > /var/lib/shorewall/.iptables-restore-input > Line 134: > -A loc2fw -p 6 --dport 8080 -m conntrack --ctorigdst ! 10.3.0.10 -j ACCEPT > > editing out this line in rules allows a start: > > REDIRECT loc 8080 tcp 80 - !10.3.0.10 > > Did I miss something along the journey?Looks like iptables-restore 1.4.1.1 is broken. That syntax is correct: /usr/sbin/iptables -m conntrack -h ... conntrack match v1.4.0 options: [!] --ctstate [INVALID|ESTABLISHED|NEW|RELATED|UNTRACKED|SNAT|DNAT][,...] State(s) to match [!] --ctproto proto Protocol to match; by number or name, eg. `tcp'' --ctorigsrc [!] address[/mask] Original source specification --ctorigdst [!] address[/mask] <==================================== Original destination specification --ctreplsrc [!] address[/mask] Reply source specification --ctrepldst [!] address[/mask] Reply destination specification [!] --ctstatus [NONE|EXPECTED|SEEN_REPLY|ASSURED|CONFIRMED][,...] Status(es) to match [!] --ctexpire time[:time] Match remaining lifetime in seconds against value or range of values (inclusive) and ursa:~ # iptables -t nat -N foo ursa:~ # iptables -t nat -A foo -m conntrack --ctorigdst ! 10.1.1.1 -j ACCEPT ursa:~ # iptables -V iptables v1.4.0 ursa:~ # What happens when you try that on your 1.4.1.1? -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
Tom Eastep wrote:> Jerry Vonau wrote: >> One a side note: >> >> Running /sbin/iptables-restore... >> iptables-restore v1.4.1.1: host/network `!'' not found >> Error occurred at line: 134 >> Try `iptables-restore -h'' or ''iptables-restore --help'' for more >> information. >> ERROR: iptables-restore Failed. Input is in >> /var/lib/shorewall/.iptables-restore-input >> Line 134: >> -A loc2fw -p 6 --dport 8080 -m conntrack --ctorigdst ! 10.3.0.10 -j >> ACCEPT >> >> editing out this line in rules allows a start: >> >> REDIRECT loc 8080 tcp 80 - !10.3.0.10 >> >> Did I miss something along the journey? > > Looks like iptables-restore 1.4.1.1 is broken. That syntax is correct:As a follow up, I inserted that rule into my own rules file and Shorewall 4.2.0 (pre)Beta3 restarted successfully. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
Tom Eastep wrote:> Jerry Vonau wrote: >...>> >> Getting the "squid in loc" to work with "loose" took a bit of effort >> but that works now. Give me a bit, I''ll have some config info that >> worked for me if you want. > > Please -- I haven''t tested that configuration. >I''ll paste together what I brewed up.>> >> One a side note: >> >> Running /sbin/iptables-restore... >> iptables-restore v1.4.1.1: host/network `!'' not found >> Error occurred at line: 134 >> Try `iptables-restore -h'' or ''iptables-restore --help'' for more >> information. >> ERROR: iptables-restore Failed. Input is in >> /var/lib/shorewall/.iptables-restore-input >> Line 134: >> -A loc2fw -p 6 --dport 8080 -m conntrack --ctorigdst ! 10.3.0.10 -j >> ACCEPT >> >> editing out this line in rules allows a start: >> >> REDIRECT loc 8080 tcp 80 - >> !10.3.0.10 >> >> Did I miss something along the journey? > > Looks like iptables-restore 1.4.1.1 is broken. That syntax is correct: > > /usr/sbin/iptables -m conntrack -h...> What happens when you try that on your 1.4.1.1? >/sbin/iptables -m conntrack -h iptables v1.4.1.1 ... conntrack match options: [!] --ctstate {INVALID|ESTABLISHED|NEW|RELATED|UNTRACKED|SNAT|DNAT}[,...] State(s) to match [!] --ctproto proto Protocol to match; by number or name, e.g. "tcp" [!] --ctorigsrc address[/mask] [!] --ctorigdst address[/mask] [!] --ctreplsrc address[/mask] [!] --ctrepldst address[/mask] Original/Reply source/destination address [!] --ctorigsrcport port [!] --ctorigdstport port [!] --ctreplsrcport port [!] --ctrepldstport port TCP/UDP/SCTP orig./reply source/destination port [!] --ctstatus {NONE|EXPECTED|SEEN_REPLY|ASSURED|CONFIRMED}[,...] Status(es) to match [!] --ctexpire time[:time] Match remaining lifetime in seconds against value or range of values (inclusive) --ctdir {ORIGINAL|REPLY} Flow direction of packet Guess it''s a bug... off to file it.. fyi: libnetfilter_conntrack-0.0.89-0.1.svn7356.fc9.i386 iptables-1.4.1.1-1.fc9.i386 2.6.25.9-76.fc9.i686 checking on updates... Jerry ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
On Sun, 2008-07-13 at 17:05 -0500, Jerry Vonau wrote:> Guess it''s a bug... off to file it.. fyi: > libnetfilter_conntrack-0.0.89-0.1.svn7356.fc9.i386 > iptables-1.4.1.1-1.fc9.i386 > 2.6.25.9-76.fc9.i686I can confirm the bug in Fedora 9: [root@fedora9 ~]# iptables -t nat -N foo [root@fedora9 ~]# iptables -t nat -A foo -m conntrack --ctorigdst ! 10.1.1.1 -j ACCEPT iptables v1.4.1.1: host/network `!'' not found Try `iptables -h'' or ''iptables --help'' for more information. [root@fedora9 ~]# cat /etc/fedora-release Fedora release 9 (Sulphur) [root@fedora9 ~]# -Tom ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
Tom Eastep wrote:> On Sun, 2008-07-13 at 17:05 -0500, Jerry Vonau wrote: > >> Guess it''s a bug... off to file it.. fyi: >> libnetfilter_conntrack-0.0.89-0.1.svn7356.fc9.i386 >> iptables-1.4.1.1-1.fc9.i386 >> 2.6.25.9-76.fc9.i686 > > I can confirm the bug in Fedora 9: > > [root@fedora9 ~]# iptables -t nat -N foo > [root@fedora9 ~]# iptables -t nat -A foo -m conntrack --ctorigdst ! > 10.1.1.1 -j ACCEPT > iptables v1.4.1.1: host/network `!'' not found > Try `iptables -h'' or ''iptables --help'' for more information. > [root@fedora9 ~]# cat /etc/fedora-release > Fedora release 9 (Sulphur) > [root@fedora9 ~]# > > -Tomfiled: https://bugzilla.redhat.com/show_bug.cgi?id=455197 Jerry ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
Tom Eastep wrote:> Jerry Vonau wrote: > >> >> Ok, is been more that a couple of days. ;-) With 4.2, is the reason >> behind the shorewall test layout, using main 999, is for backwards >> compatibility? > > Yes. > >> >> Getting the "squid in loc" to work with "loose" took a bit of effort >> but that works now. Give me a bit, I''ll have some config info that >> worked for me if you want. > > Please -- I haven''t tested that configuration. >Just a quick summary.. shorewall show routing Shorewall 4.2.0-Beta3 Routing at shore.wp.shawcable.net - Sun Jul 13 09:45:40 CDT 2008 Routing Rules 0: from all lookup local 999: from all lookup main hold-over from old config 9996: from all to 10.3.0.168 iif eth0 lookup ETH1 9997: from all to 10.5.0.0/24 lookup main 9998: from all to 10.10.0.2 lookup main not needed route in main.. 10000: from all fwmark 0x1 lookup ETH1 10001: from all fwmark 0x2 lookup ETH0 10004: from all fwmark 0x5 lookup bal 20000: from 24.78.192.197 lookup ETH1 note the lack of "from 10.3.0.75" from loose, must use fwmarks. 32767: from all lookup default Table bal: ## just an old name ;) 10.3.0.11 dev eth0 scope link src 10.3.0.75 default via 10.3.0.11 dev eth0 src 10.3.0.75 Table default: default nexthop via 24.78.192.1 dev eth1 weight 10 nexthop via 10.3.0.1 dev eth0 weight 1 Table ETH0: 10.3.0.1 dev eth0 scope link src 10.3.0.75 default via 10.3.0.1 dev eth0 src 10.3.0.75 Table ETH1: 24.78.192.1 dev eth1 scope link src 24.78.192.197 default via 24.78.192.1 dev eth1 src 24.78.192.197 Table local: local 10.10.0.1 dev tun0 proto kernel scope host src 10.10.0.1 broadcast 127.255.255.255 dev lo proto kernel scope link src 127.0.0.1 broadcast 10.3.0.0 dev eth0 proto kernel scope link src 10.3.0.75 broadcast 24.78.193.255 dev eth1 proto kernel scope link src 24.78.192.197 broadcast 24.78.192.0 dev eth1 proto kernel scope link src 24.78.192.197 local 24.78.192.197 dev eth1 proto kernel scope host src 24.78.192.197 local 10.3.0.75 dev eth0 proto kernel scope host src 10.3.0.75 broadcast 10.3.0.255 dev eth0 proto kernel scope link src 10.3.0.75 broadcast 127.0.0.0 dev lo proto kernel scope link src 127.0.0.1 local 127.0.0.1 dev lo proto kernel scope host src 127.0.0.1 local 127.0.0.0/8 dev lo proto kernel scope host src 127.0.0.1 Table main: 10.10.0.2 dev tun0 proto kernel scope link src 10.10.0.1 10.3.0.0/24 dev eth0 proto kernel scope link src 10.3.0.75 10.5.0.0/24 via 10.10.0.2 dev tun0 24.78.192.0/23 dev eth1 proto kernel scope link src 24.78.192.197 providers: ETH1 1 1 - $EXTDEV 24.78.192.1 track,balance=10 ETH0 2 2 - $LOCDEV $LOCGW track,balance=1 bal 5 5 - $LOCDEV $SQUID loose The squid box is a subject for another thread, two gateway on the same lan using ip-aliases on one interface, hence the 10,11 addresses below. .10 using the same .1 gateway, while .11 using .75(here) as a gateway. tcrules: ... old test stuff 1:P 10.3.0.0/24 0.0.0.0/0 all - - - 1:P 10.3.0.10 0.0.0.0/0 all - - - 2:P 10.3.0.11 0.0.0.0/0 all - - - 5:P 10.3.0.10 0.0.0.0/0 tcp - 3128 - 5:P 10.3.0.11 0.0.0.0/0 tcp - 3128 - 5:P 10.3.0.11 0.0.0.0/0 tcp - 80 - 5:P 10.3.0.0/24 0.0.0.0/0 tcp 80 - - And just to hedge my bet, start: iptables -t mangle -A PREROUTING -i eth0 -d ! 10.3.0.75 -p tcp --dport 80 -j MARK --set-mark=5 Both the above dport 80 rules appear to be marking, so I''m not sure which is working. I''ll bet both, just on different chains. shorewall show mangle Shorewall 4.2.0-Beta3 Mangle Table at shore.wp.shawcable.net - Sun Jul 13 10:23:27 CDT 2008 Counters reset Sun Jul 13 00:53:30 CDT 2008 Chain PREROUTING (policy ACCEPT 79707 packets, 20M bytes) pkts bytes target prot opt in out source destination 59464 16M CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 connmark match !0x0/0xff CONNMARK restore mask 0xff 832 65741 routemark all -- eth1 * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff 19409 4053K routemark all -- eth0 * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff 27293 6072K tcpre all -- eth1 * 0.0.0.0/0 0.0.0.0/0 52412 14M tcpre all -- eth0 * 0.0.0.0/0 0.0.0.0/0 2 100 tcpre all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff 10721 1860K MARK tcp -- eth0 * 0.0.0.0/0 !10.3.0.75 tcp dpt:80 MARK xset 0x5/0xffffffff Chain INPUT (policy ACCEPT 8298 packets, 944K bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 66784 packets, 17M bytes) pkts bytes target prot opt in out source destination 66784 17M tcfor all -- * * 0.0.0.0/0 0.0.0.0/0 Chain OUTPUT (policy ACCEPT 10180 packets, 1397K bytes) pkts bytes target prot opt in out source destination 10006 1385K CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 connmark match !0x0/0xff CONNMARK restore mask 0xff 174 11495 tcout all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff Chain POSTROUTING (policy ACCEPT 76241 packets, 18M bytes) pkts bytes target prot opt in out source destination 76241 18M tcpost all -- * * 0.0.0.0/0 0.0.0.0/0 Chain routemark (2 references) pkts bytes target prot opt in out source destination 832 65741 MARK all -- eth1 * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x1/0xffffffff 19409 4053K MARK all -- eth0 * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x2/0xffffffff 20241 4119K CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 mark match !0x0/0xff CONNMARK save mask 0xff Chain tcfor (1 references) pkts bytes target prot opt in out source destination Chain tcout (1 references) pkts bytes target prot opt in out source destination 0 0 MARK icmp -- * * 0.0.0.0/0 !10.3.0.0/24 MARK xset 0x1/0xffffffff 0 0 MARK tcp -- * * 0.0.0.0/0 !10.3.0.0/24 tcp dpt:443 MARK xset 0x1/0xffffffff 1 60 MARK tcp -- * * 0.0.0.0/0 !10.3.0.0/24 tcp dpt:21 MARK xset 0x1/0xffffffff 50 3269 MARK udp -- * * 10.3.0.75 !10.3.0.0/24 udp dpt:53 MARK xset 0x1/0xffffffff 69 4602 MARK udp -- * * 24.78.192.197 !10.3.0.0/24 udp dpt:53 MARK xset 0x1/0xffffffff 0 0 MARK tcp -- * * 10.3.0.75 !10.3.0.0/24 multiport dports 80,53 MARK xset 0x1/0xffffffff 0 0 MARK tcp -- * * 10.3.0.75 !10.3.0.0/24 multiport dports 80,53 MARK xset 0x1/0xffffffff 7 404 MARK tcp -- * * 24.78.192.197 !10.3.0.0/24 multiport dports 80,53 MARK xset 0x1/0xffffffff 0 0 MARK tcp -- * * 24.78.192.197 !10.3.0.0/24 tcp dpt:25 MARK xset 0x1/0xffffffff 0 0 MARK tcp -- * * 10.3.0.75 !10.3.0.0/24 tcp dpt:25 MARK xset 0x1/0xffffffff Chain tcpost (1 references) pkts bytes target prot opt in out source destination Chain tcpre (3 references) pkts bytes target prot opt in out source destination 1222 64826 MARK tcp -- * * 10.3.0.0/24 0.0.0.0/0 tcp dpt:23 MARK xset 0x2/0xffffffff 1222 64826 MARK tcp -- * * 10.3.0.0/24 0.0.0.0/0 tcp dpt:23 MARK xset 0x2/0xffffffff 27293 6072K MARK all -- eth1 * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x1/0xffffffff 0 0 MARK tcp -- eth0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:110 MARK xset 0x1/0xffffffff 52411 14M MARK all -- * * 10.3.0.0/24 0.0.0.0/0 MARK xset 0x1/0xffffffff 2979 185K MARK all -- * * 10.3.0.10 0.0.0.0/0 MARK xset 0x1/0xffffffff 3 252 MARK all -- * * 10.3.0.11 0.0.0.0/0 MARK xset 0x2/0xffffffff 0 0 MARK tcp -- * * 10.3.0.10 0.0.0.0/0 tcp spt:3128 MARK xset 0x5/0xffffffff 0 0 MARK tcp -- * * 10.3.0.11 0.0.0.0/0 tcp spt:3128 MARK xset 0x5/0xffffffff 0 0 MARK tcp -- * * 10.3.0.11 0.0.0.0/0 tcp spt:80 MARK xset 0x5/0xffffffff 10721 1860K MARK tcp -- * * 10.3.0.0/24 0.0.0.0/0 tcp dpt:80 MARK xset 0x5/0xffffffff Byte count is the same, I''ll strip out the start file and see what that brings.. Got to go, off to work for now.. Jerry ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08
Jerry Vonau wrote:> >>> Getting the "squid in loc" to work with "loose" took a bit of effort >>> but that works now. Give me a bit, I''ll have some config info that >>> worked for me if you want. >> Please -- I haven''t tested that configuration. >> > Just a quick summary..> > providers: > ETH1 1 1 - $EXTDEV 24.78.192.1 track,balance=10 > ETH0 2 2 - $LOCDEV $LOCGW track,balance=1 > bal 5 5 - $LOCDEV $SQUID loose > > > The squid box is a subject for another thread, two gateway on the same > lan using ip-aliases on one interface, hence the 10,11 addresses below. > .10 using the same .1 gateway, while .11 using .75(here) as a gateway. > > tcrules: > ... old test stuff > 1:P 10.3.0.0/24 0.0.0.0/0 all - - - > 1:P 10.3.0.10 0.0.0.0/0 all - - - > 2:P 10.3.0.11 0.0.0.0/0 all - - - > 5:P 10.3.0.10 0.0.0.0/0 tcp - 3128 - > 5:P 10.3.0.11 0.0.0.0/0 tcp - 3128 - > 5:P 10.3.0.11 0.0.0.0/0 tcp - 80 - > 5:P 10.3.0.0/24 0.0.0.0/0 tcp 80 - - > > And just to hedge my bet, start: > > iptables -t mangle -A PREROUTING -i eth0 -d ! 10.3.0.75 -p tcp --dport > 80 -j MARK --set-mark=5 > > Both the above dport 80 rules appear to be marking, so I''m not sure > which is working. I''ll bet both, just on different chains.Yes -- I forget now why I recommended that entry in start rather than one in tcrules; at the time I wrote that HOWTO, something needed for that rule was missing. I''m still unclear what you had to change to get the Squid stuff working... -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer''s challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/