NOTE: this mail started as a help call, but I''ve been wrinting it
through several days, and several tests that lead more to an
investigation that to an actual question. I think it''s somewhat
usefull,
specially if thewre are comments on why the different cases didn''t work
as spected. take in account that I didn''t try to fix its redaction.
I have two ISPs, one reliable but restrictive and one unreliable but
open. I also have a internal network to which I should offer best effort
service. let me explain.
the reliable network, let''s call it UNC, only offers access to
http/s and ssh. the unreliable one. Arnet, lets me access any port, but
the BW is much smaller and often the link goes down.
so what I want is to route all traffic, including the firewall
generated, through Arnet (even if it''s unreliable), except http/s and
ssh, which goes though UNC, including traffic to the UNC network itself.
my current configuration is:
# shorewall.conf
[...]
MARK_IN_FORWARD_CHAIN=No
[...]
# providers
# NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY
OPTIONS COPY
UNC 1 0x100 main eth0 200.16.29.111
track,fallback,balance -
Arnet 2 0x200 main eth2 10.0.0.2
track -
# tcrules
# MARK SOURCE DEST PROTO PORT CLIENT PORTS USER
TEST
# everything through the Arnet link
0x200 - -
# UNC networks go through the UNC link
0x100 - 200.16.0.0/16
0x100 - 172.16.0.0/16
# also http/s and ssh connections generated from the internal network.
# the source does not include the FW itself! yes, I know, se below.
0x100 172.16.21.0/24 - tcp 80
0x100 172.16.21.0/24 - tcp 443
0x100 172.16.21.0/24 - tcp 22
this worked mostly fine, except for the cuts in the Arnet link,
which were espected. you can find shorewall info and mtr test here[1][2],
the files with ''works'' in its name. fw-<machine> are mtr
originated from
the fw, russell is in the UNC network and www.arnet.com.ar is in the
Arnet network. client-<machine> are originated in the 172.16.21.0/24
network.
now, there are a couple of problems with this setup.
today the UNC network went down, so I tried to redirect all traffic
through the Arnet link. what I did was to simply comment out all the
marks for the UNC link and restart shorewall.
for some reason it didn''t work. what I was using for testing is a
simple «mtr -c3 -r <machine>». it was trying to go out through the UNC
link. this is, I think, because the default gw in the main routing table
is the router that links us to UNC.
tom sugested to change the first rule to:
# providers
# MARK SOURCE DEST PROTO PORT CLIENT PORTS USER
TEST
0x200:P - -
0x200 $FW -
so packets originating from the FW were also marked. trying those
mtr commands in the FW again didn''t work at all, even when taking out
the fallback option in the UNC entry in providers. they did work from
the client. I quickly rolled back to the old setup. what I''m doing
wrong
here, and how''s it is supposed to be done?
tom also mentioned that both providers should have the balance
option, but I don''t understand exactly what the fallback, balance and
loose options do. it works as it worked before, as the mtr show (the
''balance'' set). but as you can see clearly balance is not
something that
I want in boths links, I just want to clearly route some traffic though
one and the rest through the other.
now, taking out the balance (the ''no_balance'' set) from
the UNC
provider sends ICMP packets to russell through the Arnet link, not the
UNC as it should be. I will stick to this.
taking out both all the ''balance'' and the
''fallback'' options work as
spected, without the first suggestion from tom. puttong back both
''balance'' but no ''fallback'' sends ICMP
packets to the Arnet network
through UNC. no fun.
as conclusion, just putting the ''track'' option on each
provider
worked as spected, even when I commented the lines in tcrules that
marked traffic for UNC, which was what I wanted. I know this is poor
man''s BGP or any another routing protocol, but I can''t do a
more complex
setup.
--
[1] http://iate.oac.uncor.edu/~mdione/shorewall/
[2] you can also find a sw.tbz with all those files.
--
(Not so) Random fortune:
Y en que lugar, habrá consuelo para mi locura,
esta ironía con que se cura, si el final es donde partí.
Y a quién llamar, a quién golpearle la puerta tan tarde,
con quién hablar cuando no hay nadie, si esta noche no puedo dormir.
-- La Renga, "El final es donde partí"
------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals.
Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com
Marcos Dione wrote:> NOTE: this mail started as a help call, but I''ve been wrinting it > through several days, and several tests that lead more to an > investigation that to an actual question. I think it''s somewhat usefull, > specially if thewre are comments on why the different cases didn''t work > as spected. take in account that I didn''t try to fix its redaction. > > I have two ISPs, one reliable but restrictive and one unreliable but > open. I also have a internal network to which I should offer best effort > service. let me explain. > > the reliable network, let''s call it UNC, only offers access to > http/s and ssh. the unreliable one. Arnet, lets me access any port, but > the BW is much smaller and often the link goes down. > > so what I want is to route all traffic, including the firewall > generated, through Arnet (even if it''s unreliable), except http/s and > ssh, which goes though UNC, including traffic to the UNC network itself. > > my current configuration is: > > # shorewall.confMarcos, I''m sorry to hear that you are having problems with Shorewall Multi-ISP support and that I may have contributed to the problem. But it is virtually impossible for me to help people who sent snippets of their configuration files that then ask "Why doesn''t it work?". The problem with configuration files is that they represent your interpretation of what is required for your particular configuration. But they don''t tell us anything about that configuration itself. And when dealing with networking issues, the details are extremely important. When I don''t have access to the details, the configuration files (especially pieces of those files) result in more questions than answers. That is why the Support Guidelines (http://www.shorewall.net/support.htm#Guidelines) ask that when you have any kind of connection problem that you send the output of ''shorewall dump'' collected as described rather than sending configuration files. The ''shorewall dump'' command has been under development since Shorewall was initially released, and its output is tailored to give us the best chance of solving your problem quickly. Maybe others on the list will be willing to dig through your post and try to understand what the problem is, I simply don''t have the time to do that. Sorry, -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com
On Wed, 2009-05-27 at 20:09 -0300, Marcos Dione wrote:> NOTE: this mail started as a help call, but I've been wrinting it > through several days, and several tests that lead more to an > investigation that to an actual question. I think it's somewhat usefull, > specially if thewre are comments on why the different cases didn't work > as spected. take in account that I didn't try to fix its redaction. > > I have two ISPs, one reliable but restrictive and one unreliable but > open. I also have a internal network to which I should offer best effort > service. let me explain. > > the reliable network, let's call it UNC, only offers access to > http/s and ssh. the unreliable one. Arnet, lets me access any port, but > the BW is much smaller and often the link goes down. > > so what I want is to route all traffic, including the firewall > generated, through Arnet (even if it's unreliable), except http/s and > ssh, which goes though UNC, including traffic to the UNC network itself. > > my current configuration is: > > # shorewall.conf > [...] > MARK_IN_FORWARD_CHAIN=No > [...] > > > # providers > # NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY > UNC 1 0x100 main eth0 200.16.29.111 track,fallback,balance - > Arnet 2 0x200 main eth2 10.0.0.2 track - > > > # tcrules > # MARK SOURCE DEST PROTO PORT CLIENT PORTS USER TEST > # everything through the Arnet link > 0x200 - - > > # UNC networks go through the UNC link > 0x100 - 200.16.0.0/16 > 0x100 - 172.16.0.0/16 > > # also http/s and ssh connections generated from the internal network. > # the source does not include the FW itself! yes, I know, se below. > 0x100 172.16.21.0/24 - tcp 80 > 0x100 172.16.21.0/24 - tcp 443 > 0x100 172.16.21.0/24 - tcp 22 > > > this worked mostly fine, except for the cuts in the Arnet link, > which were espected. you can find shorewall info and mtr test here[1][2], > the files with 'works' in its name. fw-<machine> are mtr originated from > the fw, russell is in the UNC network and www.arnet.com.ar is in the > Arnet network. client-<machine> are originated in the 172.16.21.0/24 > network. > > now, there are a couple of problems with this setup. > > today the UNC network went down, so I tried to redirect all traffic > through the Arnet link. what I did was to simply comment out all the > marks for the UNC link and restart shorewall. > > for some reason it didn't work. what I was using for testing is a > simple «mtr -c3 -r <machine>». it was trying to go out through the UNC > link. this is, I think, because the default gw in the main routing table > is the router that links us to UNC. > > tom sugested to change the first rule to: > > # providers > # MARK SOURCE DEST PROTO PORT CLIENT PORTS USER TEST > 0x200:P - - > 0x200 $FW - > > so packets originating from the FW were also marked. trying those > mtr commands in the FW again didn't work at all, even when taking out > the fallback option in the UNC entry in providers. they did work from > the client. I quickly rolled back to the old setup. what I'm doing wrong > here, and how's it is supposed to be done? >from http://www.shorewall.net/MultiISP.html Regardless of whether you have masqueraded hosts or not, the following entries are required in /etc/shorewall/masq if you plan to redirect connections from the firewall using entries in /etc/shorewall/tcrules or if you specify balance on your providers. #INTERFACE SUBNET ADDRESS eth0 130.252.99.27 206.124.146.176 eth1 206.124.146.176 130.252.99.27 Those entries ensure that traffic originating on the firewall and redirected via packet marks always has the source IP address corresponding to the interface that it is routed out of. Your missing the SNAT entries for the firewall itself... eth0 10.0.0.2 200.16.29.66 eth2 200.16.29.66 10.0.0.2> tom also mentioned that both providers should have the balance > option, but I don't understand exactly what the fallback, balance and > loose options do. it works as it worked before, as the mtr show (the > 'balance' set). but as you can see clearly balance is not something that > I want in boths links, I just want to clearly route some traffic though > one and the rest through the other. > > now, taking out the balance (the 'no_balance' set) from the UNC > provider sends ICMP packets to russell through the Arnet link, not the > UNC as it should be. I will stick to this. > > taking out both all the 'balance' and the 'fallback' options work as > spected, without the first suggestion from tom. puttong back both > 'balance' but no 'fallback' sends ICMP packets to the Arnet network > through UNC. no fun. > > as conclusion, just putting the 'track' option on each provider > worked as spected, even when I commented the lines in tcrules that > marked traffic for UNC, which was what I wanted. I know this is poor > man's BGP or any another routing protocol, but I can't do a more complex > setup. >Hope it helps, Jerry ------------------------------------------------------------------------------ Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
Marcos Dione wrote:> > my current configuration is:Although I''m not going to try to solve your issues, I''ll offer some comments on your configuration.> > # shorewall.conf > [...] > MARK_IN_FORWARD_CHAIN=No > [...] > > > # providers > # NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY > UNC 1 0x100 main eth0 200.16.29.111 track,fallback,balance - > Arnet 2 0x200 main eth2 10.0.0.2 track -The COPY column should not be empty -- it should contain a list of your interfaces other than eth0 and eth2. Depending on what version of Shorewall you are running (you failed to mention that -- it''s the FIRST thing that we look at in the output of ''shorewall dump''), there are problems with COPY and you may need to include the interface named in the INTERFACE column in that list.> > > # tcrules > # MARK SOURCE DEST PROTO PORT CLIENT PORTS USER TEST > # everything through the Arnet link > 0x200 - - > > # UNC networks go through the UNC link > 0x100 - 200.16.0.0/16 > 0x100 - 172.16.0.0/16 > > # also http/s and ssh connections generated from the internal network. > # the source does not include the FW itself! yes, I know, se below. > 0x100 172.16.21.0/24 - tcp 80 > 0x100 172.16.21.0/24 - tcp 443 > 0x100 172.16.21.0/24 - tcp 22 > > > this worked mostly fine, except for the cuts in the Arnet link, > which were espected. you can find shorewall info and mtr test here[1][2], > the files with ''works'' in its name. fw-<machine> are mtr originated from > the fw, russell is in the UNC network and www.arnet.com.ar is in the > Arnet network. client-<machine> are originated in the 172.16.21.0/24 > network. > > now, there are a couple of problems with this setup. > > today the UNC network went down, so I tried to redirect all traffic > through the Arnet link. what I did was to simply comment out all the > marks for the UNC link and restart shorewall. > > for some reason it didn''t work. what I was using for testing is a > simple «mtr -c3 -r <machine>». it was trying to go out through the UNC > link. this is, I think, because the default gw in the main routing table > is the router that links us to UNC. > > tom sugested to change the first rule to: > > # providers > # MARK SOURCE DEST PROTO PORT CLIENT PORTS USER TEST > 0x200:P - - > 0x200 $FW - > > so packets originating from the FW were also marked. trying those > mtr commands in the FW again didn''t work at all, even when taking out > the fallback option in the UNC entry in providers. they did work from > the client. I quickly rolled back to the old setup. what I''m doing wrong > here, and how''s it is supposed to be done?I personally set ''optional'' in /etc/shorewall/interfaces and simply restart Shorewall when an interface goes down. But then, you have complete access to my configuration. It''s the ''Complete Example'' on the Shorewall Multi-ISP page. And you will note that I don''t set ''balance'' on both interfaces.> > tom also mentioned that both providers should have the balance > option, but I don''t understand exactly what the fallback, balance and > loose options do.The Shorewall Multi-ISP page explains them -- if you have explicit questions, I''m happy to answer them.> it works as it worked before, as the mtr show (the > ''balance'' set). but as you can see clearly balance is not something that > I want in boths links, I just want to clearly route some traffic though > one and the rest through the other. >Sigh -- the FAQ talks about that. But I guess no one believes those FAQ entries...> now, taking out the balance (the ''no_balance'' set) from the UNC > provider sends ICMP packets to russell through the Arnet link, not the > UNC as it should be. I will stick to this. >Sorry -- I don''t understand that. But it sounds like you are missing the critical entries in /etc/shorewall/masq described on the Multi-ISP page. But you didn''t include that file at all in your post. See why we want the output of ''shorewall dump''? I see that Jerry Vonau just posted with that suggestion also.> taking out both all the ''balance'' and the ''fallback'' options work as > spected, without the first suggestion from tom. puttong back both > ''balance'' but no ''fallback'' sends ICMP packets to the Arnet network > through UNC. no fun. >Again -- we have no idea what that means. But it sounds like the missing masq entries again.> as conclusion, just putting the ''track'' option on each provider > worked as spected, even when I commented the lines in tcrules that > marked traffic for UNC, which was what I wanted. I know this is poor > man''s BGP or any another routing protocol, but I can''t do a more complex > setup. >-Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com
On Wed, May 27, 2009 at 05:31:14PM -0700, Tom Eastep wrote:> Marcos Dione wrote: > > NOTE: this mail started as a help call, but I''ve been wrinting it > > through several days, and several tests that lead more to an > > investigation that to an actual question. I think it''s somewhat usefull, > > specially if thewre are comments on why the different cases didn''t work > > as spected. take in account that I didn''t try to fix its redaction. > > > > I have two ISPs, one reliable but restrictive and one unreliable but > > open. I also have a internal network to which I should offer best effort > > service. let me explain. > > > > the reliable network, let''s call it UNC, only offers access to > > http/s and ssh. the unreliable one. Arnet, lets me access any port, but > > the BW is much smaller and often the link goes down. > > > > so what I want is to route all traffic, including the firewall > > generated, through Arnet (even if it''s unreliable), except http/s and > > ssh, which goes though UNC, including traffic to the UNC network itself. > > > > my current configuration is: > > > > # shorewall.conf > > Marcos, > > I''m sorry to hear that you are having problems with Shorewall Multi-ISP > support and that I may have contributed to the problem. But it is > virtually impossible for me to help people who sent snippets of their > configuration files that then ask "Why doesn''t it work?".I knew this is the situation. I mentioned your suggestions to express that I was lost and that didn''t knew exactly what I was doing.> That is why the Support Guidelines > (http://www.shorewall.net/support.htm#Guidelines) ask that when you have > any kind of connection problem that you send the output of ''shorewall > dump'' collected as described rather than sending configuration files.I didn''t attach them, but I put then in the URL I sent. I didn''t want to flood the list with several dumps. -- (Not so) Random fortune: When I first started working with sendmail, I was convinced that the cf file had been created by someone bashing their head on the keyboard. After a week, I realised this was, indeed, almost certainly the case. -- Unknown ------------------------------------------------------------------------------ Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com
Marcos Dione wrote:> > I didn''t attach them, but I put then in the URL I sent. I didn''t > want to flood the list with several dumps. >My apologies -- I missed the links. I''ll try to look at them over the weekend. -Tom -- Tom Eastep \ The ultimate result of shielding men from the effects of Shoreline, \ folly is to fill the world with fools. Washington, USA \ -- Herbert Spencer ------------------------------------------------------------------------ http://www.shorewall.net ------------------------------------------------------------------------------ Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com
Marcos Dione wrote:> On Wed, May 27, 2009 at 05:31:14PM -0700, Tom Eastep wrote: >> Marcos Dione wrote: >>> NOTE: this mail started as a help call, but I''ve been wrinting it >>> through several days, and several tests that lead more to an >>> investigation that to an actual question. I think it''s somewhat usefull, >>> specially if thewre are comments on why the different cases didn''t work >>> as spected. take in account that I didn''t try to fix its redaction. >>> >>> I have two ISPs, one reliable but restrictive and one unreliable but >>> open. I also have a internal network to which I should offer best effort >>> service. let me explain. >>> >>> the reliable network, let''s call it UNC, only offers access to >>> http/s and ssh. the unreliable one. Arnet, lets me access any port, but >>> the BW is much smaller and often the link goes down. >>> >>> so what I want is to route all traffic, including the firewall >>> generated, through Arnet (even if it''s unreliable), except http/s and >>> ssh, which goes though UNC, including traffic to the UNC network itself. >>> >>> my current configuration is: >>> >>> # shorewall.conf >> Marcos, >> >> I''m sorry to hear that you are having problems with Shorewall Multi-ISP >> support and that I may have contributed to the problem. But it is >> virtually impossible for me to help people who sent snippets of their >> configuration files that then ask "Why doesn''t it work?". > > I knew this is the situation. I mentioned your suggestions to > express that I was lost and that didn''t knew exactly what I was doing. > >> That is why the Support Guidelines >> (http://www.shorewall.net/support.htm#Guidelines) ask that when you have >> any kind of connection problem that you send the output of ''shorewall >> dump'' collected as described rather than sending configuration files. > > I didn''t attach them, but I put then in the URL I sent. I didn''t > want to flood the list with several dumps. >Marcos, Can you tell me which directories in the tarball correspond to which failure scenarios above? That would certainly help. The naming seems confusing because the configuration in the directory called no_balance has a balanced default route: default nexthop via 200.16.29.111 dev eth0 weight 1 nexthop via 10.0.0.2 dev eth2 weight 1 Thanks, -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com
Tom Eastep wrote:> Marcos Dione wrote: >> On Wed, May 27, 2009 at 05:31:14PM -0700, Tom Eastep wrote: >>> Marcos Dione wrote: >>>> NOTE: this mail started as a help call, but I''ve been wrinting it >>>> through several days, and several tests that lead more to an >>>> investigation that to an actual question. I think it''s somewhat usefull, >>>> specially if thewre are comments on why the different cases didn''t work >>>> as spected. take in account that I didn''t try to fix its redaction. >>>> >>>> I have two ISPs, one reliable but restrictive and one unreliable but >>>> open. I also have a internal network to which I should offer best effort >>>> service. let me explain. >>>> >>>> the reliable network, let''s call it UNC, only offers access to >>>> http/s and ssh. the unreliable one. Arnet, lets me access any port, but >>>> the BW is much smaller and often the link goes down. >>>> >>>> so what I want is to route all traffic, including the firewall >>>> generated, through Arnet (even if it''s unreliable), except http/s and >>>> ssh, which goes though UNC, including traffic to the UNC network itself. >>>> >>>> my current configuration is: >>>> >>>> # shorewall.conf >>> Marcos, >>> >>> I''m sorry to hear that you are having problems with Shorewall Multi-ISP >>> support and that I may have contributed to the problem. But it is >>> virtually impossible for me to help people who sent snippets of their >>> configuration files that then ask "Why doesn''t it work?". >> I knew this is the situation. I mentioned your suggestions to >> express that I was lost and that didn''t knew exactly what I was doing. >> >>> That is why the Support Guidelines >>> (http://www.shorewall.net/support.htm#Guidelines) ask that when you have >>> any kind of connection problem that you send the output of ''shorewall >>> dump'' collected as described rather than sending configuration files. >> I didn''t attach them, but I put then in the URL I sent. I didn''t >> want to flood the list with several dumps. >> > > Marcos, > > Can you tell me which directories in the tarball correspond to which > failure scenarios above? That would certainly help. > > The naming seems confusing because the configuration in the directory > called no_balance has a balanced default route: > > default > nexthop via 200.16.29.111 dev eth0 weight 1 > nexthop via 10.0.0.2 dev eth2 weight 1 >Possibly part of the problem here is the way that Shorewall handles the default route in the main table. 1. When there are entries in /etc/shorewall/providers, ''shorewall start'' saves the default route that it finds in the main table. 2. When there are ''balance'' providers, the default route in the main table is replaced by both ''shorewall start'' and ''shorewall restart''. 3. When there are no ''balance'' providers, Shorewall does not change the default route in the main table during ''shorewall restart''. 4. In all cases, ''shorewall stop'' restores the default route saved at the last ''shorewall start''. So if you have a configuration that specifies ''balance'' on one or more providers and you then remove ''balance'' entirely, ''shorewall restart'' will *not* remove the balanced route in the main table. To do that, you need ''shorewall stop; shorewall start''. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com
Marcos Dione wrote:> NOTE: this mail started as a help call, but I''ve been wrinting it > through several days, and several tests that lead more to an > investigation that to an actual question. I think it''s somewhat usefull, > specially if thewre are comments on why the different cases didn''t work > as spected. take in account that I didn''t try to fix its redaction. > > I have two ISPs, one reliable but restrictive and one unreliable but > open. I also have a internal network to which I should offer best effort > service. let me explain. > > the reliable network, let''s call it UNC, only offers access to > http/s and ssh. the unreliable one. Arnet, lets me access any port, but > the BW is much smaller and often the link goes down. > > so what I want is to route all traffic, including the firewall > generated, through Arnet (even if it''s unreliable), except http/s and > ssh, which goes though UNC, including traffic to the UNC network itself. > > my current configuration is:From the documentation you posted on the web, we see: Shorewall 4.0.15 Dump at samba1 - mié may 27 18:39:44 ART 2009 Shorewall-shell 4.0.15 So you are running Shorewall-shell 4.0.15. I would advise you to migrate to Shorewall-perl at your first opportunity. I see that you are also running Debian (probably Lenny): /proc/version = Linux version 2.6.26-2-486 (Debian 2.6.26-15) (dannf@debian.org) (gcc version 4.1.3 20080704 (prerelease) (Debian 4.1.2-25)) #1 Thu Mar 26 00:13:41 UTC 2009 In the next major Debian release (Squeeze), Shorewall-shell will not be supported at all. It is much better to convert to Shorewall-perl at your leisure rather than as part of a distribution upgrade. So I further urge you to use Roberto Sanchez''s Lenny repository and run Shorewall 4.2 rather than Shorewall 4.0. Roberto is the Debian Shorewall maintainer and information about his repository is available at http://people.connexer.com/~roberto/debian/> > # shorewall.conf > [...] > MARK_IN_FORWARD_CHAIN=No > [...] > > > # providers > # NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY > UNC 1 0x100 main eth0 200.16.29.111 track,fallback,balance -The ''fallback'' option is not supported by Shorewall-shell and each time that you start or restart Shorewall, it is issuing a WARNING indicating that ''fallback'' is an unknown option. From http://www.shorewall.net/MultiISP.html: --------------------------------------------------------------------------- fallback[=weight] (Added in Shorewall-perl 4.2.5) Indicates that a default route through the provider should be added to the default routing table (table 253). If a weight is given, a balanced route is added with the weight of this provider equal to the specified weight. If the option is given without a weight, an separate default route is added through the provider''s gateway; the route has a metric equal to the provider''s NUMBER. The option is ignored with a warning message if USE_DEFAULT_RT=Yes in shorewall.conf. --------------------------------------------------------------------------- If a feature is added in Shorewall-perl 4.2.5, you cannot expect to use it in Shorewall-shell 4.0.15. The current version in Roberto''s repository is 4.2.9.> Arnet 2 0x200 main eth2 10.0.0.2 track - > > > # tcrules > # MARK SOURCE DEST PROTO PORT CLIENT PORTS USER TEST > # everything through the Arnet link > 0x200 - - > > # UNC networks go through the UNC link > 0x100 - 200.16.0.0/16 > 0x100 - 172.16.0.0/16 > > # also http/s and ssh connections generated from the internal network. > # the source does not include the FW itself! yes, I know, se below. > 0x100 172.16.21.0/24 - tcp 80 > 0x100 172.16.21.0/24 - tcp 443 > 0x100 172.16.21.0/24 - tcp 22 > > > this worked mostly fine, except for the cuts in the Arnet link, > which were espected. you can find shorewall info and mtr test here[1][2], > the files with ''works'' in its name. fw-<machine> are mtr originated from > the fw, russell is in the UNC network and www.arnet.com.ar is in the > Arnet network. client-<machine> are originated in the 172.16.21.0/24 > network. > > now, there are a couple of problems with this setup. > > today the UNC network went down, so I tried to redirect all traffic > through the Arnet link. what I did was to simply comment out all the > marks for the UNC link and restart shorewall.Once you have upgraded to Shorewall-perl 4.2.9, I recommend a setup similar to my own which you can see at http://www.shorewall.net/MultiISP.html#Complete. My requirements are similar to yours. I have two providers: Comcast and Avvanta. Comcast is very fast and I prefer to use it for all connections that don''t need to go out through Avvanta. Avvanta has 5 static IP addresses so I use it for my web server, mail server, etc. I set ''balance'' on Comcast and ''fallback'' on Avvanta. See below for an explaination of what those options do.> > for some reason it didn''t work. what I was using for testing is a > simple «mtr -c3 -r <machine>». it was trying to go out through the UNC > link. this is, I think, because the default gw in the main routing table > is the router that links us to UNC.Continuing with my recommendation. If you set the ''optional'' interface option on both of your interfaces and use something like ''lsm'' or ''swping'' to monitor the links, when one of the links goes down then Shorewall is automatically restarted and the dead link is omitted from the Multi-ISP routing configuration.> > tom sugested to change the first rule to: > > # providers > # MARK SOURCE DEST PROTO PORT CLIENT PORTS USER TEST > 0x200:P - - > 0x200 $FW - > > so packets originating from the FW were also marked. trying those > mtr commands in the FW again didn''t work at all, even when taking out > the fallback option in the UNC entry in providers. they did work from > the client. I quickly rolled back to the old setup. what I''m doing wrong > here, and how''s it is supposed to be done?As both Jerry and I have pointed out, you are missing critical entries in /etc/shorewall/masq. It appears that you currently have: #INTERFACE SOURCE eth0 eth1 eth2 eth1 I recommend that you change that to read: #INTERFACE SOURCE eth0 0.0.0.0/0 eth2 0.0.0.0/0> > tom also mentioned that both providers should have the balance > option, but I don''t understand exactly what the fallback, balance and > loose options do. it works as it worked before, as the mtr show (the > ''balance'' set). but as you can see clearly balance is not something that > I want in boths links, I just want to clearly route some traffic though > one and the rest through the other.Again, I recommend that once you are running Shorewall-perl 4.2.9 that you set ''fallback'' on UNC and ''balance'' on Arnet. ''balance'' causes Shorewall to insert a ''balanced'' default route in the main routing table; the route is balanced among all providers with ''balance''. ''fallback'' causes a balanced route to be inserted into the default routing table. The route is balanced among all providers with the ''fallback'' option. If you take my recommendations, then when Arnet is down, there will be no default route at all in the main table. All traffic to the Internet will then pass to the default table and will be routed through UNC. ''loose'' determines if Shorewall creates routing rules that send certain traffic from the firewall out of an interface. If ''loose'' is not set and an application binds a socket to a particular IP address on the firewall, then connections using that socket will go out through the interface to whose IP address the socket is bound. Simply put, if you specify the local IP address that the application is to use then it will use the right interface for outgoing connections. If ''loose'' is set, then Shorewall omits those routing rules. If a connection using the socket goes out a different interface, then the entries in /etc/shorewall/masq will cause the SOURCE IP address in the connections packets to be rewritten correctly.> > now, taking out the balance (the ''no_balance'' set) from the UNC > provider sends ICMP packets to russell through the Arnet link, not the > UNC as it should be. I will stick to this. > > taking out both all the ''balance'' and the ''fallback'' options work as > spected, without the first suggestion from tom. puttong back both > ''balance'' but no ''fallback'' sends ICMP packets to the Arnet network > through UNC. no fun.I''ve already explained in a separate post that turning ''balance'' on and off can lead to unexpected results unless you ''stop;start'' rather than ''restart''. And since your version of Shorewall doesn''t support ''fallback'', the setting of that option is immaterial in your case.> > as conclusion, just putting the ''track'' option on each provider > worked as spected, even when I commented the lines in tcrules that > marked traffic for UNC, which was what I wanted. I know this is poor > man''s BGP or any another routing protocol, but I can''t do a more complex > setup. >Hope this helps. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp as they present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com